Slideshow explaining VDev, zpool, ZIL and L2ARC for noobs!

Discussion in 'New to FreeNAS?' started by cyberjock, Jul 13, 2012.

  1. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 13, 2012

    Slideshow explaining VDev, zpool, ZIL and L2ARC and other newbie mistakes!

    I've put together a Powerpoint presentation(and PDF) that gives some useful info for newbies to FreeNAS. I decided to create this slideshow because in the last 5 months I've been on this forum I've seen a lot of people confused about vdevs, zpools, zils, l2arcs, etc. Hopefully we can put to rest alot of the confusion once and for all.

    We get a large amount of duplicate threads with the same questions being asked every other day. Personally, we get them so often I decided to stop answering them and decided a better use of my time would be to create this presentation. I literally read every thread and every post that goes on the forum. So if I don't answer either the answer is in the thread or the answer is found in this presentation or FreeNAS manual. Answering every 3rd thread with "Consult the manual" gets a little old after a while and I have better uses for my time.

    This presentation also contains a lot of information that is explained in a little more detail for new users. It includes many common errors newbies make and can save you some heartache. If you are brand new to FreeBSD, I recommend reading the manual cover to cover. There are a lot of recommendations throughout the manual, and they are typically there because they are an error trap for many people.

    I've saved this in Powerpoint because I have some animations in the slideshow. I'm not sure what other formats would work. If you would like this in another format that supports animations please let me know and I'll see what I can do. Currently I provide this in a powerpoint presentation and PDF. The PDF has no animations therefore the Powerpoint is preferred.

    I'll try to keep it updated as necessary. If this presentation helps you please "like" it so others can see that it really is worth the time to read!

    Updated as of: August 18, 2014

    Powerpoint: https://dl.dropboxusercontent.com/u/57989017/FreeNAS Guide 9.2.1.pptx.zip (This is the preferred format because it includes animations)

    PDF via Google Docs: https://drive.google.com/file/d/0BzHapVfrocfwbXYxcGgycEIzNG8/edit?usp=sharing

    PDF: https://dl.dropboxusercontent.com/u/57989017/FreeNAS Guide 9.2.1.pdf.zip
    Lomik, mjh74, Bjoern and 78 others like this.
  2. Offline

    James FreeNAS Core Team

    Member Since:
    May 24, 2011
    Messages:
    295
    Message Count:
    295
    Likes Received:
    21
    Trophy Points:
    18
    Occupation:
    iXsystems, Inc.
    Location:
    San Jose, CA
    Home page:
    James, Jul 13, 2012

    Is it possible to generate a PDF for the non-animated portions for those of us who don't use Windows?
  3. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 13, 2012

    Link has been added for PDF.
    Code and Nindustries like this.
  4. Offline

    bfishernc

    Member Since:
    Jun 29, 2012
    Messages:
    30
    Message Count:
    30
    Likes Received:
    0
    Trophy Points:
    0
    bfishernc, Jul 13, 2012

    nicely done - confirmed everything I've learned so far (as a noob) :)
  5. Offline

    paleoN FreeNAS Guru

    Member Since:
    Apr 22, 2012
    Messages:
    1,403
    Message Count:
    1,403
    Likes Received:
    15
    Trophy Points:
    38
    paleoN, Jul 13, 2012

    First, let me say good job. I would also like to officially request a web-based version of this. You can even use animated gifs and no external program needed to view it.

    Now, on to the critiques.

    • [size=-2]
      VDevs are allocated in RAID formats such as RAID0(striped), RAIDZ and RAIDZ2

      [/size]
    You left out mirrors.


    • [size=-2]
      You cannot add more hard drives to a VDev once it is created.
      [/size]

    Keep it. It's a very important point. Also, it's inaccurate in the one exception that you can turn a stripe into a mirror. Perhaps asterisk it?


    • [size=-2]
      Failure of a ZIL will result in a loss of data.
      [/size]

    I'd change this to a loss of the entire pool. Which is what will happen with ZFS v15, what we use now.

    I'd add to the L2ARC section that using an L2ARC will decrease the amount of RAM available to the ARC cache. This is particularly noticeable with a large L2ARC.

    Perhaps add the caveats that most people do not need a ZIL, L2ARC, or see a benefit from them and the single best increase in performance will come from maxing out their RAM. If they even need to do that.
  6. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 14, 2012

    I didn't leave out mirrors, at least not intentionally. I was simply providing a few examples.

    I didn't know that you can turn a stripe into a mirror. I didn't know you could turn a VDev stripe into a mirror. I'll have to look into that. Can I find more information in the manual?

    As for the failure of a ZIL, will it's failure cause the pool to go offline and be unrecoverable? Or is the data that is in the ZIL and not committed to the pool simply lost? I've never played with ZILs myself because I don't see a need for them in any of my circumstances.

    Can you explain the cause/effect of the L2ARC changing the amount of RAM allocated to the ARC? I believe you, I've just never heard of this before and I'm very curious to know more about this phenomenon and the though process behind it.

    I am trying to shy away from comments about what to recommend people. Each setup can potentially have different situations. I'd prefer to show what things you shouldn't do for data security than things you shouldn't do for performance reasons. Of course, I'm arguing this idea with myself because far too many people have first posts with comments like "I have 2GB of RAM and 30TB of storage and I get kernel panics all the time". My concern is that the presentation could suddenly include situations such as "recommend using an Intel NIC because they are so cheap, they work, and they have excellent performance". Maybe I should add a slide of other random tidbits of information for noobs at the end with comments like these.
  7. Offline

    paleoN FreeNAS Guru

    Member Since:
    Apr 22, 2012
    Messages:
    1,403
    Message Count:
    1,403
    Likes Received:
    15
    Trophy Points:
    38
    paleoN, Jul 14, 2012

    Just all the others. ;)

    From the zpool man page:

    And from the ZFS Administration Guide see Example 4–6.

    The entire pool unrecoverable:


    This will be the expected behavior with ZFS v28 which will be in FreeNAS 8.3 as it's in FreeBSD 8.3.

    Also, the ZIL is for sync writes only.

    ZFS has to keep track of what's in the L2ARC tables which of necessity requires RAM. I believe it's all or almost all metadata which also means you may hit the metadata cache limit for ZFS depending on how much RAM & how large the L2ARC. I've read that it uses 25MB of RAM per 1GB of L2ARC, but I remember thinking that was high & the source wasn't the best. I will have to see if I can track down some decent references, as I'm no expert on the L2ARC, and maybe even bookmark them this time.

    Also, the L2ARC is for random reads mostly static data, i.e. databases. You will see little to no benefit for streaming workloads.

    Understandable and you're right each setup is different. It's just the ZIL and L2ARC are specifically for performance. Most noobs seem to think, "I have a SSD. I'll just add it to the pool and it will be faster." Where in truth it depends on their workload and even then it's usually much cheaper to max the RAM on their motherboard than get an SSD. Which will also get them superior performance.
  8. Offline

    paleoN FreeNAS Guru

    Member Since:
    Apr 22, 2012
    Messages:
    1,403
    Message Count:
    1,403
    Likes Received:
    15
    Trophy Points:
    38
    paleoN, Jul 14, 2012

    Found a good source.

    Now it's not nearly as bad as that. Most of us won't be using 2TB L2ARC and have only zvols or tuned the zfs record size to 8K.

    A much more typical scenario using a 300 GB L2ARC:
    Once again you would never actually see the 111GB. For an 8K record size like above the L2ARC would use about 6.8GB of RAM. A 32K record size would drop it down to 1.7GB.

    It's been my understanding/assumption that ZFS auto-tunes the L2ARC. Which would mean if you had a SSD that was "too big" or you had some crazy record sizes you would just never fully use it. The impression I have now is the "auto-tuning" is done when arc_meta_limit is hit. By default arc_meta_limit is 25% of arc_max. Using a L2ARC would fill this. What's more I believe there will be memory contention between ARC & L2ARC metadata. While the ARC would win most of it, the L2ARC would still use some.
  9. Offline

    calgarychris Newbie

    Member Since:
    Aug 27, 2011
    Messages:
    91
    Message Count:
    91
    Likes Received:
    2
    Trophy Points:
    8
    calgarychris, Jul 15, 2012

    Thanks for putting together the presentation, very helpful from this noobie's prespective!
  10. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 16, 2012

    Ok, I've added all of your recommendations Paleon and the files have been updated accordingly :).

    Edit: I actually added lots of small things. I also made the comment that emulation using VMWare works great for experimenting if you don't want to dedicate hardware. Anyone know if Openbox works with FreeNAS for experimenting?
  11. Offline

    Joshua Parker Ruehlig FreeNAS Guru

    Member Since:
    Dec 5, 2011
    Messages:
    1,443
    Message Count:
    1,443
    Likes Received:
    62
    Trophy Points:
    48
    Location:
    Sacramento, California, United States
    Joshua Parker Ruehlig, Jul 16, 2012

    Thanks for doing this noobsauce. You might want to specifiy that the zil will cause pool failure now, but in the future it will not. Once FreeNAS is on ZFS Pool Version 19 or up the zil can be removed, or crash without loss of data. (Only 95% sure on this though)
    bmcclure937 likes this.
  12. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 16, 2012

    You are correct. My intention is to keep it accurate. Right now it is accurate. When ZFS is upgraded, then I'll revise it accordingly. :)
  13. Offline

    paleoN FreeNAS Guru

    Member Since:
    Apr 22, 2012
    Messages:
    1,403
    Message Count:
    1,403
    Likes Received:
    15
    Trophy Points:
    38
    paleoN, Jul 16, 2012

    Looking good noobsauce80. :) It's paleoN :D by the way.

    I quite like the Additional thoughts for noobs... section.

    Now I don't mean to harp on this and I know I got this changed but...
    • [size=-2]
      You cannot add more hard drives to a VDev once it is created. The only exception is converting a zpool from RAID0 to RAID1. Consult the FREENAS manual for more information.

      [/size]
    It's lost some of it's impact. "You cannot add more hard drives to a VDev once it is created", is a vital and poorly understood point. Perhaps have the exception on a line below and a smaller font or something?

    This didn't make the cut: ;)
    The L2ARC was specifically designed for small random reads. The 2nd part is also true without any tuning.
  14. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 17, 2012

    To be honest.. I actually considered not including the exception. Who really will use that one exception? It's hard to believe someone would create a RAID0, then convert it to a RAID1. For noobs, do we REALLY want to be giving out a bunch of exceptions? I mean.. I dunno. I'm very torn on it. It almost seems better to say you can't add more drives than to try to give this exception. I can see it now.. the forum filling up with people that thought that "while they converted it to a RAID1 they'd add a few more hard drives to make it bigger".

    Part of me prefers 100% accuracy, but part of me wants to leave out this exception because it's kind of odd and I don't know who would actually use it. I could see maybe ZFS masters using this "feature", but noobs? No way. Personally, I just accept that you can't add more drives to a vdev and leave it at that. Although it might work, I'd prefer to not deal with this exception just in case it backfires. ;)

    Edit: I'll incorporate the latest changes tomorrow. Thanks for all of the feedback so far paleo*N*. :) My darn spell checker/grammar checker keeps trying to fix it as a typo when I type the upper case. LOL.
  15. Offline

    BryanOB

    Member Since:
    May 27, 2012
    Messages:
    10
    Message Count:
    10
    Likes Received:
    0
    Trophy Points:
    0
    Occupation:
    Retired Automotive Engineer
    Location:
    Michigan, USA
    BryanOB, Jul 17, 2012

    noobsauce80, thank you for posting the slideshow. It did help me and I particularly liked the animations.

    However, I bet when you started this project you had no idea that it might mushroom into something larger, because of requests like the following :)

    Define the VDEV - e.g. Virtual Device.

    There are two other important aspects that could help others (me too).

    1) The relationship with hard drives having 4KB sectors, the ZFS 128KB stripe size and the number of drives in a VDEV.
    re: http://forums.freenas.org/showthread.php?158-What-number-of-drives-are-allowed-in-a-RAIDZ-config

    2) Page (18) of slideshow: The new VDEV can use a different size than the old VDEV. Please clairfy.
    To me, it implied that an unbalanced Zpool was OK. E.g. A Zpool with......
    VDEV1 with RAIDZ2 and 6 drives
    VDEV2 with RAIDZ1 and 3 drives

    Thanks,
    Bryan
  16. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 19, 2012

    Ok. I've added ZFS striping info and updated Page 18 of the slideshow based on BryanOB and paleoN's feedback.
  17. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 21, 2012

    Updated slideshow again. Now states it is up to date with 8.2-RELEASE-p1.

    Also added the following to slide 26:

    -Always keep a backup of your FreeNAS configuration(section 2.5 of the FreeNAS manual). Especially before you update FreeNAS.
  18. Offline

    BryanOB

    Member Since:
    May 27, 2012
    Messages:
    10
    Message Count:
    10
    Likes Received:
    0
    Trophy Points:
    0
    Occupation:
    Retired Automotive Engineer
    Location:
    Michigan, USA
    BryanOB, Jul 21, 2012

    Thanks for the update. I will review the PPT file , and reply if I can contribute again.

    WOW! version 8.2 is released? I didn't know that. Just a couple days ago I visited the download page - didn't notice it. OK! It was just released 21 hours ago.

    Anyway I've just built my server and I'm in the process of configuring it. I certainly hope that they fixed the broken UPS service.
    Anyway, I will try the upgrade process - thanks for reminding me to backup the configuration.
    I'm not concerned about trashing any data, because I haven't reached that far yet. :)

    Regards,
    Bryan
  19. Offline

    Stephens

    Member Since:
    Jun 19, 2012
    Messages:
    497
    Message Count:
    497
    Likes Received:
    1
    Trophy Points:
    0
    Stephens, Jul 24, 2012

    Good job. I found it surprising that in example 3 a HDD crash in VDEV1 would cause a loss of the whole ZSPOOL (including VDEV2), so that was nice to learn. I'm staying away from multiple VDEV's, though and going with multiple NAS's. Among other reasons, it gives me some hardware redundancy (like if a motherboard or power supply goes out on NAS1, I still have NAS2).

    Reading the PPT, I found "Adding VDEVs" confusing. Specifically... "It is not recommended that you [...] NOT add..." I could be wrong, but I think this means to say "It's recommended that you [...] NOT add..." But the lesson pointed to is really important.
  20. Offline

    cyberjock Forum Guard Dog/Admin

    Member Since:
    Mar 25, 2012
    Messages:
    13,777
    Message Count:
    13,777
    Likes Received:
    714
    Trophy Points:
    113
    cyberjock, Jul 28, 2012

    At Stephens request I have fixed the slide. Although I think it was grammatically correct, it could be confusing(especially to users that speak english as a second language). I also changed the virtual machines to VMWare ESXi and VirtualBox. VMWare Workstation 8.0.4 seems to not play very well with FreeNAS(at least for me). If someone else confirms it works fine I'll readd it and consider it a fluke on my end.

    Edit: If an Admin could change the thread title to "Slideshow explaining VDev, zpool, ZIL and L2ARC and other newbie mistakes!" I'd appreciate it. I tried to change it, but either I can't find it or I don't have the rights to. Thanks!

Share This Page