[wellylug] CF card lifetime

[Richard Hector]

On Tue, 2011-03-22 at 12:35 +1300, Ewen McNeill wrote:
> On 2011-03-22 11:52 , Sam Vilain wrote:
> > On 21/03/11 22:56, Richard Hector wrote:
> >> I've recently started using a Soekris box with a CF card as my home
> >> firewall.  [...]
> >> Does anyone know if there are tools to
> >> let me know how close to the end I am? It doesn't support S.M.A.R.T.
> >
> > Hey Richard, apprently YAFFS / YAFFS2 supports wear-level counts at the
> > filesystem layer - try formatting it as that.  And make double sure you
> > take regular backups as with any unusual filesystem :-).
> 
> As I understand it YAFFS is designed to work on raw flash (NAND/NOR), 
> and provides the flash translation layer, etc (like JFFS does).  That's 
> presumably how it can provide wear level counts.  By contrast a CF card 
> has its own firmware in it which provides the flash translation layer, 
> and that's what knows about the wear level counts of individual 
> sections.  So at first glance I'd say that YAFFS wouldn't work on a CF card.
> 
> There was an excellent article in LWN a few weeks back about CF/SD/etc 
> cards and the wear levelling they provide:
> 
> http://lwn.net/Articles/428584/
> 
> A _few_ of them do provide either SMART (mostly SSD, rather than CF/SD) 
> or some proprietary tool for getting at the wear levelling counts (IIRC 
> there's a link in the comments to some article that talks about 
> vendor-supported real world testing, and gives wear counts).
> 
> That same article also talks about how most of the CF/SD cards have 
> firmware that is optimised (to some lesser or greater degree) for 
> FAT(32) file systems, and can perform from poorly to terribly when used 
> with other file systems (eg, quite a bit of wear multiplication when 
> writing less-than-full-erase-blocks in areas outside the position it 
> expects the FAT to be).  There is considerably variance between vendors.
> (The comments also point out that the wear levelling algorithms in SSDs 
> tend to be fairly good these days -- they have a big enough CPU to run 
> the firmware to use sane algorithms -- but the ones on resource limited 
> CF/SD cards are often very primitive.)
> 
> For a firewall, the typical mode of operation with a CF/SD card is to 
> boot and run from a RAM disk which holds everything, possibly with a 
> read-only partition for things which are used only occasionally. 
> Logging and anything else involving writes is either to a RAM disk, or 
> via the network (eg, syslog).  Most of the dedicated-firewall embedded 
> Linux/etc systems work like that.  Assuming that's the case the number 
> of writes to the CF should be "almost none" at which point wear 
> levelling ceases to be an issue (which is, of course, one of the major 
> reasons they do it that way).
> 
> If you're treating your CF as a regular hard drive, mounting with 
> typical linux file systems (and even worse haven't mounted with 
> "noatime" -- so every read is also a write!), then I'd guess the 
> lifetime would be relatively poor.  But even so the write load on a 
> _firewall_ shouldn't be that high, unless you're logging packets 
> traversing the system.  So I'd guess the CF should be good for many 
> months, possibly years.  (If you are logging non trivial amounts of 
> traffic directly to CF, you probably want to stop now!)

Yep. I'm running ordinary squeeze, having looked at various embedded
options and not liked them much - I know how to apply security updates
on debian, is the main one.

I have, however, told syslog to send everything elsewhere, so the only
logging left local is stuff that doesn't use syslog (aptitude, dpkg,
wtmp/utmp, ... forgot to put my work key on it so I can't check now).
And I have set noatime :-)

> PS: Aside from SMART and some vendor proprietary (possibly Microsoft 
> Windows only) tools I'm not aware of any other way to peak into what the 
> embedded firmware is doing for wear levelling.  So you essentially get 
> no warning that it's no longer possible to erase some of the blocks on 
> the CF.  (And depending on the firmware in the CF it may either silently 
> work around that for you, as modern hard spinning rust hard drives work 
> around bad blocks, or -- more likely on cheap CF -- randomly give you 
> corrupt blocks.  Fun.)

Ok. I was afraid of that :-(

Thanks,

Richard