I have almost the same system, with the Atom D510 and 5x 2TB disks and idling my power is about 46w and during a scrub its just under 60w if I remember correctly.Originally Posted by bdacasc
I have almost the same system, with the Atom D510 and 5x 2TB disks and idling my power is about 46w and during a scrub its just under 60w if I remember correctly.
The reads you see there are performed on my Windows WD5000AAKX disks, set in RAID1. After the FreeNAS upgrade, if I transfer a file from local disk to network disk (mapped to FreeNAS through CIFS) I get about 50MB/sec as transfer speed. I used to get 110MB/sec before, so something changed in a negative way. Not very happy now.
Run this command as Administrator in your Windows system, to make sure the disks are not the bottleneck:
Code:C:\Windows\system32> winsat disk -drive c
Last edited by TECK; 04-11-2012 at 07:07 PM.
I figured out that those results were from windows. I'm not really happy with my results either, especially knowing what you used to be able to get. I know your CPU is a bit faster than mine, but I should be able to get a lot closer to what you had than I am. I need to take a break for awhile, my eyes are getting blurry and my brain getting foggy
If anyone has a fix or knows what might be the cause of data speed loss, please let us know. Since I moved from 8.0.2, I lost half of the transfer speed I used to get.
See if this works for you. I'm getting about 50MB/s writes and 60MB/s reads with peaks of 70MB/s. I don't think I've got it the best it can be yet, the speeds seem to drop off a little during the transfers, but I think I'm getting closer. Let me know what you get. My laptop disk can only do ~68MB/s, so I can't do much faster than that anyway.
For anyone else reading this, these settings are for an Atom motherboard, D510 processor, 8GB of RAM, 5 disk raidz2.
CIFS Auxilary Settings (You need to enable Advance Settings checkbox for FreeNAS "System->Settings" to see this). After enabling that checkbox, go to Services->Control Services and click on the wrench icon next to CIFS.
For the loader.conf (Tunables in GUI), depending on your RAM etc. these seem to work well at the moment:Code:socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=2097152 SO_SNDBUF=2097152 # 65536 or 131072 seem to work ok for above values also, experiment read raw = yes write raw = yes max xmit = 65536 getwd cache = yes
Note: Dividing some of the large numbers by 1048576 (1MB) will give you the value in Megabytes
Code:vfs.zfs.prefetch_disable 0 (Setting to 1 cuts my speeds in half) vfs.zfs.arc_max 5074190464 (For 8GB of RAM) kern.ipc.nmbclusters 10000 vfs.zfs.txg.write_limit_override 1073741824 (For 8GB of RAM) vfs.zfs.vdev.max_pending 1 (8 is also a recommended value) vfs.zfs.vdev.min_pending 1 (2 is also a recommended value) kern.ipc.somaxconn 384 TCP listen queue kern.ipc.nsfbufs 150000 Sendfile buffers - Used if "sendfile" is enabled for CIFS net.inet.tcp.inflight.enable 1 TCP Bandwidth Delay Product net.inet.tcp.recvbuf_auto 1 net.inet.tcp.sendbuf_auto 1 net.inet.tcp.delayed_ack 0 net.inet.tcp.sendbuf_max 2097152 kern.ipc.maxsockbuf 2097152 net.inet.tcp.recvbuf_max 2097152
Last edited by ProtoSD; 04-19-2012 at 09:26 PM.
You can get rid of those options, they are default values:
read raw = yes
write raw = yes
max xmit = 65536
getwd cache = yes
Now I remember adding these settings into aux params:
socket options = TCP_NODELAY IPTOS_LOWDELAY IPTOS_THROUGHPUT
while enabling the Large RW support, sendfile2 and AIO. Right now, I get 75MB/sec writes to network disk without any custom loader.conf values. Personally, I never fiddled with that file. However the 75MB/sec write starts dropping after 1 minute of active transfer and keeps dropping up to a miserable 48MB/sec, for a 10GB file. Before I used to get about 90MB/sec writes, dropping at 75MB/sec lowest.
I also get writes to local disk at 70MB/sec steady all the way to the end. Bye bye 115MB settings I used to get before...
Last edited by TECK; 04-19-2012 at 10:01 PM.
I can tell you're really unhappy about this. This is really the first time I've looked into the performance of my system in more detail.
I'm going to disable those Samba setting you mentioned, but I also have AIO disabled. I think I saw somewhere that AIO doesn't work right on FreeBSD anyway.
Like I said, I don't think I've got it pefect yet, just better than the default settings which are terrible.
I did share these settings with another user that was getting terrible speeds and he said his speeds are fantastic now, 90+GB/s.
Did you try to set your prefetch to 0? I thought with memory over 4GB it was supposed to be 1, but when I set it to one my transfers literally are about half, which is what you're saying.
My local disk speeds are ~320MB/s, so I know I should be able to do better. I also get better speeds with medium sized files, like 500-700MB.
I'll keep testing and post back if I make any significant progress.
Well, I get better results with AIO enabled, no idea why.
Also, do you think is good to fiddle with loader.conf file? I always tune sysctl on my Unix boxes... but I know nothing about FreeBSD. Before I was getting top results without any tuneup changes.
Remember that I have a 64GB SSD used for cache, not sure how it affects the tuneup.
You can put the settings from sysctl.conf or loader.conf into either file and they will work. Since FreeNAS 8.03 you can add settings for those files from the GUI. I just put everything into the "Loaders" section except my jail stuff which goes into the Sysctl section just for clarity.
There are a couple of settings related to cache, but I don't think they're required. The nice thing about entering the settings from the GUI is there is a "Enable" checkbox so you can test stuff and turn it off or on using the Enable checkbox. The only hassle is having to reboot to get certain settings to take effect.
Are you using LAGG?
No, I don't use LAGG. I don't see anywhere the Loaders section, into Settings gui.
Reply With Quote
