Slideshow explaining VDev, zpool, ZIL and L2ARC for noobs!

Discussion in 'New to FreeNAS?' started by cyberjock, Jul 13, 2012.

Page 1 of 14
  1. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    Slideshow explaining VDev, zpool, ZIL and L2ARC and other newbie mistakes!

    I've put together a Powerpoint presentation(and PDF) that gives some useful info for newbies to FreeNAS. I decided to create this slideshow because in the last 5 months I've been on this forum I've seen a lot of people confused about vdevs, zpools, zils, l2arcs, etc. Hopefully we can put to rest alot of the confusion once and for all.

    We get a large amount of duplicate threads with the same questions being asked every other day. Personally, we get them so often I decided to stop answering them and decided a better use of my time would be to create this presentation. I literally read every thread and every post that goes on the forum. So if I don't answer either the answer is in the thread or the answer is found in this presentation or FreeNAS manual. Answering every 3rd thread with "Consult the manual" gets a little old after a while and I have better uses for my time.

    This presentation also contains a lot of information that is explained in a little more detail for new users. It includes many common errors newbies make and can save you some heartache. If you are brand new to FreeBSD, I recommend reading the manual cover to cover. There are a lot of recommendations throughout the manual, and they are typically there because they are an error trap for many people.

    I've saved this in Powerpoint because I have some animations in the slideshow. I'm not sure what other formats would work. If you would like this in another format that supports animations please let me know and I'll see what I can do. Currently I provide this in a powerpoint presentation and PDF. The PDF has no animations therefore the Powerpoint is preferred.

    I'll try to keep it updated as necessary. If this presentation helps you please "like" it so others can see that it really is worth the time to read!​

    Updated as of: May 2, 2016

    Powerpoint: https://dl.dropboxusercontent.com/u/57989017/9.10/FreeNAS Guide 9.10.pptx.zip (This is the preferred format because it includes animations)

    This version is currently unavailable.

    PDF via Google Docs: https://drive.google.com/file/d/0BzHapVfrocfwblFvMVdvQ2ZqTGM/view?usp=sharing

    PDF: https://dl.dropboxusercontent.com/u/57989017/9.10/FreeNAS Guide 9.10.pdf.zip
     
    Last edited by a moderator: May 4, 2017
    cyberjock, Jul 13, 2012
    #1
    ctag, expediter, Boiler and 238 others said thanks for this.
  2. James

    James Team iX iXsystems

    Joined:
    May 24, 2011
    Messages:
    300
    Thanks Received:
    23
    Trophy Points:
    26
    Occupation:
    iXsystems, Inc.
    Location:
    San Jose, CA
    Home Page:
    Is it possible to generate a PDF for the non-animated portions for those of us who don't use Windows?
     
    James, Jul 13, 2012
    #2
  3. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    Link has been added for PDF.
     
    cyberjock, Jul 13, 2012
    #3
    fikus-kukis, Code and Nindustries said thanks for this.
  4. bfishernc

    bfishernc Newbie

    Joined:
    Jun 29, 2012
    Messages:
    30
    Thanks Received:
    0
    Trophy Points:
    4
    nicely done - confirmed everything I've learned so far (as a noob) :)
     
    bfishernc, Jul 13, 2012
    #4
  5. paleoN

    paleoN FreeNAS Guru

    Joined:
    Apr 22, 2012
    Messages:
    1,403
    Thanks Received:
    20
    Trophy Points:
    46
    First, let me say good job. I would also like to officially request a web-based version of this. You can even use animated gifs and no external program needed to view it.

    Now, on to the critiques.

    • [size=-2]
      VDevs are allocated in RAID formats such as RAID0(striped), RAIDZ and RAIDZ2

      [/size]
    You left out mirrors.


    • [size=-2]
      You cannot add more hard drives to a VDev once it is created.
      [/size]

    Keep it. It's a very important point. Also, it's inaccurate in the one exception that you can turn a stripe into a mirror. Perhaps asterisk it?


    • [size=-2]
      Failure of a ZIL will result in a loss of data.
      [/size]

    I'd change this to a loss of the entire pool. Which is what will happen with ZFS v15, what we use now.

    I'd add to the L2ARC section that using an L2ARC will decrease the amount of RAM available to the ARC cache. This is particularly noticeable with a large L2ARC.

    Perhaps add the caveats that most people do not need a ZIL, L2ARC, or see a benefit from them and the single best increase in performance will come from maxing out their RAM. If they even need to do that.
     
    paleoN, Jul 13, 2012
    #5
  6. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    I didn't leave out mirrors, at least not intentionally. I was simply providing a few examples.

    I didn't know that you can turn a stripe into a mirror. I didn't know you could turn a VDev stripe into a mirror. I'll have to look into that. Can I find more information in the manual?

    As for the failure of a ZIL, will it's failure cause the pool to go offline and be unrecoverable? Or is the data that is in the ZIL and not committed to the pool simply lost? I've never played with ZILs myself because I don't see a need for them in any of my circumstances.

    Can you explain the cause/effect of the L2ARC changing the amount of RAM allocated to the ARC? I believe you, I've just never heard of this before and I'm very curious to know more about this phenomenon and the though process behind it.

    I am trying to shy away from comments about what to recommend people. Each setup can potentially have different situations. I'd prefer to show what things you shouldn't do for data security than things you shouldn't do for performance reasons. Of course, I'm arguing this idea with myself because far too many people have first posts with comments like "I have 2GB of RAM and 30TB of storage and I get kernel panics all the time". My concern is that the presentation could suddenly include situations such as "recommend using an Intel NIC because they are so cheap, they work, and they have excellent performance". Maybe I should add a slide of other random tidbits of information for noobs at the end with comments like these.
     
    cyberjock, Jul 14, 2012
    #6
    clugo633 said thanks for this.
  7. paleoN

    paleoN FreeNAS Guru

    Joined:
    Apr 22, 2012
    Messages:
    1,403
    Thanks Received:
    20
    Trophy Points:
    46
    Just all the others. ;)

    From the zpool man page:

    And from the ZFS Administration Guide see Example 4–6.

    The entire pool unrecoverable:


    This will be the expected behavior with ZFS v28 which will be in FreeNAS 8.3 as it's in FreeBSD 8.3.

    Also, the ZIL is for sync writes only.

    ZFS has to keep track of what's in the L2ARC tables which of necessity requires RAM. I believe it's all or almost all metadata which also means you may hit the metadata cache limit for ZFS depending on how much RAM & how large the L2ARC. I've read that it uses 25MB of RAM per 1GB of L2ARC, but I remember thinking that was high & the source wasn't the best. I will have to see if I can track down some decent references, as I'm no expert on the L2ARC, and maybe even bookmark them this time.

    Also, the L2ARC is for random reads mostly static data, i.e. databases. You will see little to no benefit for streaming workloads.

    Understandable and you're right each setup is different. It's just the ZIL and L2ARC are specifically for performance. Most noobs seem to think, "I have a SSD. I'll just add it to the pool and it will be faster." Where in truth it depends on their workload and even then it's usually much cheaper to max the RAM on their motherboard than get an SSD. Which will also get them superior performance.
     
    paleoN, Jul 14, 2012
    #7
  8. paleoN

    paleoN FreeNAS Guru

    Joined:
    Apr 22, 2012
    Messages:
    1,403
    Thanks Received:
    20
    Trophy Points:
    46
    Found a good source.

    Now it's not nearly as bad as that. Most of us won't be using 2TB L2ARC and have only zvols or tuned the zfs record size to 8K.

    A much more typical scenario using a 300 GB L2ARC:
    Once again you would never actually see the 111GB. For an 8K record size like above the L2ARC would use about 6.8GB of RAM. A 32K record size would drop it down to 1.7GB.

    It's been my understanding/assumption that ZFS auto-tunes the L2ARC. Which would mean if you had a SSD that was "too big" or you had some crazy record sizes you would just never fully use it. The impression I have now is the "auto-tuning" is done when arc_meta_limit is hit. By default arc_meta_limit is 25% of arc_max. Using a L2ARC would fill this. What's more I believe there will be memory contention between ARC & L2ARC metadata. While the ARC would win most of it, the L2ARC would still use some.
     
    paleoN, Jul 14, 2012
    #8
  9. calgarychris

    calgarychris FreeNAS Experienced

    Joined:
    Aug 27, 2011
    Messages:
    113
    Thanks Received:
    2
    Trophy Points:
    21
    Thanks for putting together the presentation, very helpful from this noobie's prespective!
     
    calgarychris, Jul 15, 2012
    #9
  10. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    Ok, I've added all of your recommendations Paleon and the files have been updated accordingly :).

    Edit: I actually added lots of small things. I also made the comment that emulation using VMWare works great for experimenting if you don't want to dedicate hardware. Anyone know if Openbox works with FreeNAS for experimenting?
     
    cyberjock, Jul 16, 2012
    #10
  11. Joshua Parker Ruehlig

    Joshua Parker Ruehlig FreeNAS Expert

    Joined:
    Dec 5, 2011
    Messages:
    5,878
    Thanks Received:
    515
    Trophy Points:
    111
    Location:
    Sacramento, California, United States
    Thanks for doing this noobsauce. You might want to specifiy that the zil will cause pool failure now, but in the future it will not. Once FreeNAS is on ZFS Pool Version 19 or up the zil can be removed, or crash without loss of data. (Only 95% sure on this though)
     
    Joshua Parker Ruehlig, Jul 16, 2012
    #11
    bmcclure937 said thanks for this.
  12. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    You are correct. My intention is to keep it accurate. Right now it is accurate. When ZFS is upgraded, then I'll revise it accordingly. :)
     
    cyberjock, Jul 16, 2012
    #12
  13. paleoN

    paleoN FreeNAS Guru

    Joined:
    Apr 22, 2012
    Messages:
    1,403
    Thanks Received:
    20
    Trophy Points:
    46
    Looking good noobsauce80. :) It's paleoN :D by the way.

    I quite like the Additional thoughts for noobs... section.

    Now I don't mean to harp on this and I know I got this changed but...
    • [size=-2]
      You cannot add more hard drives to a VDev once it is created. The only exception is converting a zpool from RAID0 to RAID1. Consult the FREENAS manual for more information.

      [/size]
    It's lost some of it's impact. "You cannot add more hard drives to a VDev once it is created", is a vital and poorly understood point. Perhaps have the exception on a line below and a smaller font or something?

    This didn't make the cut: ;)
    The L2ARC was specifically designed for small random reads. The 2nd part is also true without any tuning.
     
    paleoN, Jul 16, 2012
    #13
  14. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    To be honest.. I actually considered not including the exception. Who really will use that one exception? It's hard to believe someone would create a RAID0, then convert it to a RAID1. For noobs, do we REALLY want to be giving out a bunch of exceptions? I mean.. I dunno. I'm very torn on it. It almost seems better to say you can't add more drives than to try to give this exception. I can see it now.. the forum filling up with people that thought that "while they converted it to a RAID1 they'd add a few more hard drives to make it bigger".

    Part of me prefers 100% accuracy, but part of me wants to leave out this exception because it's kind of odd and I don't know who would actually use it. I could see maybe ZFS masters using this "feature", but noobs? No way. Personally, I just accept that you can't add more drives to a vdev and leave it at that. Although it might work, I'd prefer to not deal with this exception just in case it backfires. ;)

    Edit: I'll incorporate the latest changes tomorrow. Thanks for all of the feedback so far paleo*N*. :) My darn spell checker/grammar checker keeps trying to fix it as a typo when I type the upper case. LOL.
     
    cyberjock, Jul 17, 2012
    #14
  15. BryanOB

    BryanOB Newbie

    Joined:
    May 27, 2012
    Messages:
    12
    Thanks Received:
    0
    Trophy Points:
    4
    Occupation:
    Retired Automotive Engineer
    Location:
    Michigan, USA
    noobsauce80, thank you for posting the slideshow. It did help me and I particularly liked the animations.

    However, I bet when you started this project you had no idea that it might mushroom into something larger, because of requests like the following :)

    Define the VDEV - e.g. Virtual Device.

    There are two other important aspects that could help others (me too).

    1) The relationship with hard drives having 4KB sectors, the ZFS 128KB stripe size and the number of drives in a VDEV.
    re: http://forums.freenas.org/showthread.php?158-What-number-of-drives-are-allowed-in-a-RAIDZ-config

    2) Page (18) of slideshow: The new VDEV can use a different size than the old VDEV. Please clairfy.
    To me, it implied that an unbalanced Zpool was OK. E.g. A Zpool with......
    VDEV1 with RAIDZ2 and 6 drives
    VDEV2 with RAIDZ1 and 3 drives

    Thanks,
    Bryan
     
    BryanOB, Jul 17, 2012
    #15
  16. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    Ok. I've added ZFS striping info and updated Page 18 of the slideshow based on BryanOB and paleoN's feedback.
     
    cyberjock, Jul 19, 2012
    #16
  17. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    Updated slideshow again. Now states it is up to date with 8.2-RELEASE-p1.

    Also added the following to slide 26:

    -Always keep a backup of your FreeNAS configuration(section 2.5 of the FreeNAS manual). Especially before you update FreeNAS.
     
    cyberjock, Jul 21, 2012
    #17
  18. BryanOB

    BryanOB Newbie

    Joined:
    May 27, 2012
    Messages:
    12
    Thanks Received:
    0
    Trophy Points:
    4
    Occupation:
    Retired Automotive Engineer
    Location:
    Michigan, USA
    Thanks for the update. I will review the PPT file , and reply if I can contribute again.

    WOW! version 8.2 is released? I didn't know that. Just a couple days ago I visited the download page - didn't notice it. OK! It was just released 21 hours ago.

    Anyway I've just built my server and I'm in the process of configuring it. I certainly hope that they fixed the broken UPS service.
    Anyway, I will try the upgrade process - thanks for reminding me to backup the configuration.
    I'm not concerned about trashing any data, because I haven't reached that far yet. :)

    Regards,
    Bryan
     
    BryanOB, Jul 21, 2012
    #18
  19. Stephens

    Stephens FreeNAS Experienced

    Joined:
    Jun 19, 2012
    Messages:
    496
    Thanks Received:
    4
    Trophy Points:
    21
    Good job. I found it surprising that in example 3 a HDD crash in VDEV1 would cause a loss of the whole ZSPOOL (including VDEV2), so that was nice to learn. I'm staying away from multiple VDEV's, though and going with multiple NAS's. Among other reasons, it gives me some hardware redundancy (like if a motherboard or power supply goes out on NAS1, I still have NAS2).

    Reading the PPT, I found "Adding VDEVs" confusing. Specifically... "It is not recommended that you [...] NOT add..." I could be wrong, but I think this means to say "It's recommended that you [...] NOT add..." But the lesson pointed to is really important.
     
    Stephens, Jul 24, 2012
    #19
  20. cyberjock

    cyberjock Moderator Moderator

    Joined:
    Mar 25, 2012
    Messages:
    19,155
    Thanks Received:
    1,788
    Trophy Points:
    161
    At Stephens request I have fixed the slide. Although I think it was grammatically correct, it could be confusing(especially to users that speak english as a second language). I also changed the virtual machines to VMWare ESXi and VirtualBox. VMWare Workstation 8.0.4 seems to not play very well with FreeNAS(at least for me). If someone else confirms it works fine I'll readd it and consider it a fluke on my end.

    Edit: If an Admin could change the thread title to "Slideshow explaining VDev, zpool, ZIL and L2ARC and other newbie mistakes!" I'd appreciate it. I tried to change it, but either I can't find it or I don't have the rights to. Thanks!
     
    cyberjock, Jul 28, 2012
    #20
    erwinvr said thanks for this.
Page 1 of 14