Thomas Henson

Isilon Quick Tips: SyncIQ Deep Dive

by 1 Comment

OneFS offers many options to customize replication policies using SyncIQ. In this episode of Isilon Quick Tips we walk through those options in deep dive into SyncIQ.

 

Transcript

hello and welcome back to another episode of ISILON on quick tips today we’re going to talk about SyncIQ and we’re going to go a little bit deeper than we have in the past the SyncIQ so before it was all about just setting up a one-time replication now let’s talk about some of the options and how we can really customize are seeking jobs so what we’re going to do is I’m going to swing over to policies and we look at a policy of already got created i’m just going to edit that policy that policy is my home shares and so this is all my corporate home directories here in Huntsville and this is something that I’m replicating to my secondary cluster so the first thing i want to talk about I want to talk about some of the differences between copy and synchronization copy is just when you’re specifically moving data from one directory to another directory you’re not caring about if the date has been deleted or if the data is merged so when you’re synchronizing that’s going to be different because synchronizing is actually going to keep your primary cluster and your secondary cluster in sync so a file has been moved to another directory inside of that directory will be replicated that way in your secondary cluster and so will delete if you have a file has been deleted your primary once the scene job completes it will be deleted on your secondary cluster as well some of the areas you would use these is if you want to have a backup of your data but you don’t want to sink it or maybe there’s certain directors you want to pull out that you want to have all of those copy that’s typically when you’re going to see them but most cases you’re going to use the synchronization so in our run job or run job is our option of windows is dropping around so we want to sync you know this is our SyncIQ policy when they’re going to kick off you have a couple different options the one we did before was our manual so that just saying hey you know just going to manually push the button and every time i do that i’m going to sink my going to start that state policy then you also have the option to do it on schedule this is the most common one used so this is says hey you know two times a day three times a day you can set it up however you want that you’re going to set a schedule that you know the state is going to be replicated you know let’s say you do at six in the morning then six in the afternoon you know you doing a weekly monthly or yearly basis another common way to run these is whenever sources modified and so you can have a source of this modified so you know think about if you move a file if you delete a fall anything like that any changes to that directory it’s going to go ahead and modify the source you do have to set a time frame around that so let’s say that you modify a file how fast you want that see you can send delay to happen a few different seconds minutes hours or days so you can say hey you know every time some modified let’s wait a few minutes and then go ahead and never replicated over and then you can also have it set up where whenever a snapshot of the source directory is taken that’s going to run that the state policy job to setting the source directory is very simple right so what directory or want to move in this case i’m moving all my data that’s in ifs is my source directory the cool thing is where you can really customize this job is not only were sent in from different directories or also can include directories or exclude directories you can come in and say hey you know all the directories under data i’m only going to move over RI salon support or you know I’m only one will move over my easy gather / or i can exclude directories and say you know what move everything that’s in data but these two directories here I salon support a nice long gather those are all administrative things that you know not really trying to replicate over the change in a lot there’s not any data in there that’s not recoverable don’t replicate those so that just gives you a lot more control so you can set you said something at a high level in the tree and then only replicate the times that you want to in that tree and not have to worry about okay you have to set up you know 15 different policies because I’ve got different datasets now you can still come back and set one or two policies to replicate over the data and then we talk about some of these advanced settings so you can actually set a priority on this and you can take it from you know a normal default policy or you can have always have it but this specific policies always going to be high level and never just make sure that that priority has been lifted on this job here and also you can set a limit on how long you’re going to keep the reports from these jobs because you know depending on how often these jobs running you start to have a lot of reports and so you have that option there let’s cancel out of this and I’m going to close out i will show you one more thing so we’re talking about setting up those on modified depending on how often those files change that’s how much bandwidth is going to go over your network so if you have some performance concerns about how often or how much data is being problem across you can actually set of performance role in these jobs and so one of the cool things that I really like about it is you can set it on a schedule so you say hey you know I really want to replicate this anytime my name is modified but there are certain business our times maybe or certain times certain days of the week that I’m really want to throttle back and say hey you know what it’s not as a bigger problem during this time you know the rest of the week let’s go ahead and have it you know open throttle there and so you can set a schedule man you can even set some bandwidth rules around the limitations and so you can kinda throttle back saying hey you know always want to be modifying that data but let’s just set a performance rule about how much bandwidth can be taken up and so that’s just a deeper dive on CQ and so you can really see how you can customize and design those same policies to fit whatever kind of rules that you want to have for replicating your data between your eyes on clusters thanks for taking the time to watch and i hope you’ll join me for another episode of ISILON on quick tips

Isilon Quick Tips: Change Password in OneFS 8.0

by Leave a Comment

Transcript

How to change the password in OneFS 8.0 . Alright . oh so we’ll start here are management console want to navigate to the access tab and then the memberships
in roles once when our memberships and rolls we can select a provider to change the admin password I’m going to use the file system from here I can just view and edit the admin you can see I have many different options here so i can change the password I can change my home directory so what I’m going to do is I’m going to add my password and I’m going to select the bin . SH from my unix shell if you don’t select something here you’ll get an error and then I’ll just save my changes and I close this out will log out and we’ll try a new password and it worked and that’s how you change the password and one of this.

Isilon Quick Tips: Setting Up One Time Sync in Isilon’s OneFS

by 1 Comment

Setting Up One Time Sync in Isilon's OneFS

Setting Up One Time Sync in Isilon’s OneFS

Transcript

Ever wanted to set up a SyncIQ job and OneFS? Well follow along in this tutorial here we are in our production cluster you can see that we have the name hints and cluster for this Isilon cluster and in DR cluster we have it named hensondr cluster and so now you can see we have two different ones that were going to try to replicate between now let’s look at some of the file shares to show us the data that we’re actually going to try to move we’re gonna be focusing on move in our research data so if we go to ifs research data our production cluster you can see we have a lot of different files but on DR site we don’t have any of those files so how can we replicate that demo data in the research folder over to our Isilon cluster we could just do a onetime sink so let’s go under the data protection sink IQ will actually navigate to the policies tab and then from here we can just create a simples SyncIQ policy so our policy name here we’re just going to name it research one time in our description will just say hey you know this is a one-time think that were moving the research folder will keep our job type is manual but we could run it on a schedule but we’re just doing a one-time snap here and so now we need to just pick our source directory so remember we’re doing that research data directory we’re going to worry about including or excluding folders we’re just going to sink all of them and now here’s what we’ll put in our target host if you remember back to our dr cluster it’s 10 . 111 that 158 . – 15 and so we’ll just internet target host name remember that your dr cluster and so that’s where we want to sink this so we’re going from production to dr and our target directory we’re just going to dump it into the IFS directory now we can just scroll down to the bottom will keep everything the same and now we are ready to create our policy and so our policy has been created but we still need to kick that job off so we can kick that job off actually just by clicking on this view edit tab and start job so you see we got a green check saying hey our policy job has been scheduled starting now if you see under the summary tab we’re starting to move data it’s going to take some time because we’re moving 10 gigabytes of data so depending on the connection speed and the other jobs that are running but after a few minutes that job will disappear from the summaries tab and you can navigate over to the reports tab and see hey the job was finished took a minute and 31 seconds now let’s verify it so we’re looking on DR cluster we can see that we have demo data folder and we have data in the folder let’s compare it with our demo data on our production cluster so research demo data we can see the folders match up so we just created our first thing I q job remembering your dr cluster all your files are read-only so if you’re going to read or try to ride any folders you can’t do that so in a fail over situation you can switch it to read and write but default it’s going to be only in a read-only mode so if you try to add a folder or any folders you’ll get an error saying that you can’t do that

Isilon Quick Tips: Setting Up NFS Export in OneFS

by Leave a Comment

Another Isilon Quick Tip, where I walk through setting up NFS export in OneFS. Setting up NFS exports is one of the baseline skills needed when working with OneFS.

 

NFS or Network File System is a protocol that allows file based access in a distributed environments. If you are familiar with Windows based systems it’s similar to the SMB protocol but mostly used in Linux/Unix environments. Chances are if you have any Linux/UNIX machines in your environment, you will have a need for using NFS exports.

When Do I Need an NFS Export?

Let’s jump into a couple use cases when you would want to mount an NFS export.

  • Suppose you need extra capacity on your local machine
  • Offload archive data to a network based file system
  • Allow for file sharing abilities for a group of users
  • Manage file access across a in a distributed environment
  • Large data transfers or access to large files across network

Setting Up NFS Export in OneFS

  1. Open OneFS WebGUI
  2. Navigate to Protocols –> UNIX Sharing (NFS)
  3. Click Create Export
  4. Select directory to be shared
  5. Click Create Export
  6. Mount NFS export on Linux/UNIX machine (see commands below)

Transcript

In this episode of Isilon Quick Tips, we’re going to focus on accessing NFS Exports from Isilon’s OneFS.

If you’re accessing Isilon from a Linux machine, you’ll want to make use of the network file system—or NFS—protocol. To do this, we’ll be using mount commands. But first, let’s set up a directory that we want to share out through an NFS export. All this will be done from OneFS web interface and a Linux command line. So, follow along.

From our Protocol tab, we’ll go to the UNIX Sharing or NFS. Within our NFS Exports, we’ll have one defaulted, and that default will be for our IFS directory. Remember, anything in that IFS directory is everything that’s in Isilon. So, that’s one that’s set up by default, but let’s set up one that is specific just to maybe our data. So, I’m going to create an export. We can select our path and we can go down as deep as we want. So, I could go into our data and do something off the home shares or some of the archive data. But I just want to set a top-level directory for just our data path and share this one out. So, I’m going to select ifs/data, and then this is all of our data in Isilon. You don’t have to set a description. It’s just good once you start managing quite a few of these. You want to be like, okay, you can look at it and say, “Hey, okay, that’s actually what this export supports.” With our permissions, we can restrict it to read-only, but we don’t want to do that because we want to be able to make this a working directory. But I will click the “Enable mount access to subdirectories.” So, we’re not only accessing that data – we’re actually accessing everything inside of data and all the subdirectories involved as well. From here, I’ll just create my export, and we get a green check, which means we’re good to go. We now have two exports available. We have one from our IFS and one from our data. So, now we’ll need to jump back into a Linux box and access this from the command line.

So, from our Linux machine, I’m just going to show my directory path. So, I’m here in the root directory and I’ve got some files here. The first thing I want to do—and one of the ways that I always troubleshoot setting up the NFS mounts—is let’s see what mounts are available. So, we’re going to run a showmount command, and what we’re expecting to see is that IFS export, and also the IFS data that we just set up. So, the syntax is just showmount -e, and it’s going to be our Isilon cluster name. So, I’ve just got an IP address for mine. All right, and just like we expected, we see our IFS data, and then our IFS, and those are both accessible to us. Now all we have to do is create a directory to put this in. So, from our root directory, I’m just going to use an mkdir, and let’s set up a directory called our data-share. Just confirm that it’s there. And now we’ll just that mount command. So, mount [Isilon cluster name]:, which export we’re going to use. Remember, we’re going to use the IFS data, but you could use the IFS and mount to all the data that’s in Isilon. Now we need to put the full path of the directory that we want to put it in. So, we just created the data-share, and then now we should be able to run LS on our data-share. And now we see that we have our data in here. So, we have our Isilon support, we have project data, we have that home share data and that archive data – all mounted here.

So, this is a quick way just to set up an NFS export from a Linux machine to your Isilon cluster. Thanks for joining me for another Isilon Quick Tip.



Isilon Quick Tips: Setting Up SmartQuotas in OneFS

by Leave a Comment

Learn how to setup SmartQuotas with this Isilon Quick tips Episode.

Transcript

Hello and welcome back to another episode of Isilon quick tips today I want to talk about SmartQuotas. Smartquotas gives you the ability to set quotas on how much data is certain directories users or groups can go through so you’ll find it under our file system and smartQuotas is a licensed feature so you do have to have it license i have it enabled on this cluster and very straightforward let’s go ahead and create a storage quota and talk about our options I go saying you have the ability to set it at the directory level and with directories you can actually do it as a directory subdirectory you can filter down just like you can we r CQ policies or your snapshots we also have the option that you do a user quota and those can be applied to all users or you can actually set it down to individual users if you want and then you also have the option to do group quarters and you can set those groups on your group users or walkthrough just setting up one on a directory quota like this is the most popular one and the one that’s most pliable a lot of use cases and so I said we can set those at the directory and sub directory level so we can come in here and set one on our hdfs directory go ahead and set that policy up so far usage account you can actually set it to include your snapshot data and then you can also set it to include that data protection overhead so this is really giving you a true value of how much storage is being taken up by this group of users are these directories and so if you’re thinking about doing it from accounting perspective where you’re charging back to your users you might want to use this kind of accounting we’re saying hey you know it’s it’s not just about the space that you’re taking up and it’s the data protection overhead how we’re protecting the data then also the snapshots of the data as well i’m going to select any of those but we are going to come down to the usage limits and so this is the area where you can actually set those limitations so you can set it up to be no usage limit and what that is that’s just an accounting feature so if you’re saying hey you know what we don’t really want to set limits on users but we do want to know from accounting perspective how much data is being using on this specific smart program in our case it would be all the data that’s being used on ifs for / hdfs directory so how much is are hadoo storage infrastructure taking up let’s talk about setting up a specific usage limit and so we have three different options with the usage limits the first one is the Advisory limit not call this the nice guy limit so if you set an advisory limit this gives you the ability to come in and let’s just say that we know we’re setting up for ten terabytes and you know our users are you know captain 10 terabytes and it’s just going to send notifications whenever they go over that limitation is not going to stop them from riding and if you’re not going to give him any kind of grace period that’s what i call this one a nice guy the next one is the kind of tough guy and that’s our soft limit and so softly that actually gives you a grace period so if you come in here our directory and we can set it just like we were in our advisory and set it to 10 terabytes and say you know what we’re actually going to give him a grace period of you know 14 days and so what that is is going to allow them 14 days through giving them notifications to clear up and get their data back under you know 10 terabytes or you know however your set it up for you to go back in and say okay you know 10 terabytes went over that you know i’m gonna give you five more terabytes or something like that but it’s us just the kind of tough guy was like okay you don’t have to stop right now but at some point in the future you’re going to and you get to set that grace period the next one is what i call the tough guy and that’s the hard limit and so the hard limits not gonna let you writing any day to past it and so you don’t come back in center by threshold and say hey you know anything over 10 but you’re just not allowed to write you have to pick up phone call us we have to you know come in here and actually take that quote and bump it up for you if you need to but it also gives you the option to show the different space available to so one of the things that i recommend if you doing this hard limit for users is to go in and let them be able to see what that hard threshold is versus the size of the cluster because you want them to be able to say hey you know he’s only ten terabytes available now there’s only you know two terabytes left for bumping up to that limit vs if they’re just looking at the size of the cluster they’re probably thinking oh you know we’re good we can keep store and then one day they’re you know they’re not able to save something to do a lot so just my recommendation and then last but not least you have the option for these notifications so you have disabled the quota notifications use you know the fall trolls or you can actually set up your own custom rules so you just go through and create your quote here now you have your quota set up for your hdfs directory well thanks for joining me in this isilon quick tips and i hope to see you again soon