MicroSD with ECC

Some MicroSD cards (mostly Transcend) claim to have an ECC controller on the chip. I was wondering how this works? Do they verify all data
written to it, and automaticly relocate the sectors to different positions in case they detect a mismatch?
6
If so, how come they can have such a high speed rating (Class 10), when they have to read back all the data that is written, and sometimes even
have to write the data twice?

2 flash microsd error-correction

Share Cite Improve this question Follow asked Jul 10 '12 at 8:34
Muis
939 1 9 22

I split off "how [can] they have such a high speed" into a separate question: What gives some flash memory a higher speed rating than other flash memory? -- I
hope you don't mind.
– davidcary
Jul 10 '12 at 19:40

3 Answers Active Oldest Votes

does [ECC] mean you cannot use 10% of the available space on the card?

5 No. The ECC does not use any of the available user data space on the card.
Instead, the ECC is stored in separate "spare areas" next to the user
data.

spare area

Many flash memory chips are designed such that, for each 512 byte sector intended for user data, there is an additional corresponding 16 byte
"spare area". (Jim Cook. "Flash memory 101: An Introduction to NAND flash".)

Those spare areas, also called "out-of-band areas" (OOB areas), are designed to be used for ECC codes.
(Eureka Technology.
"NAND Flash FAQ".
"What is spare column in NAND flash?".
)
( Is flash chip capacity really limited to the powers of 2? )

In theory, a person could program the flash memory controller directly attached to these flash memory chips to use those "spare area" bits for
whatever he wanted.

A few flash file systems directly connected to these flash memory chips use that "spare area" in sophisticated ways to store file system data and
more complex error correction codes, instead of Hamming ECC data -- object IDs, wear-leveling data, etc. (Wikipedia: YAFFS).
(Compared to the
simple "Hamming" system described below, such file systems have some other approach to detecting and dealing with bit errors -- Reed-
Solomon codes, RAID-like multi-chip systems, or etc.).

SD/MMC cards have a tiny flash memory controller inside them, between the SD/MMC interface pins and the actual flash memory chips.
( see a b
c for photos of SD card internals, showing the small controller chip and the much larger flash memory chip.).

ECC write

how come they can have such a high speed rating

When your computer or microcontroller sends the "write a 512 byte sector" command to the SD/MMC card, the sector is erased and then written.
There's not room here to discuss the things manufacturers do to make flash faster, so I'll stick to explaining why ECC cards are not noticeably
slower than a (hypothetical) card without ECC.

My understanding is that all SD/MMC controllers should at least write Hamming ECC codes to the spare area for each block, unless they use use
some more sophisticated error-correction system.
("ST AN1823: Error Correction Code in NAND Flash Memories")
(Is there some way to tell
whether the card was really doing that or not?)

The flash memory chip is designed so the entire page (user data + spare area) is erased at once.
This erase would take the same amount of time
on a flash chip without ECC, so the ECC doesn't slow erases down.

The controller calculates the ECC data, and blindly(*) writes the user data and the ECC data to the flash. Hamming codes can be calculated "at wire
speed" while the user data is coming through the SD/MMC interface or while the user data is being sent to the flash card, so the ECC calculations
don't slow the writes down.

The controller does have to write an "extra" 16 bytes of data for each 512 byte sector, but that is not noticeably slower than a hypothetical flash
chip without ECC: 1/32 ~= 3% slower.

(*) SD card controllers don't bother reading back the data during a write to see if it worked.
They only detect problems much later, while handling
a "read" command.
Alas, I hear rumors of unscrupulous people taking perfectly good 1 GB cards and reprogramming the controller into thinking
there's actually 4 GB of flash connected to it.
The resulting fake/counterfeit/forged card seems to work fine for reads and writes until the first 1 GB
is filled up; it even seems to work fine during (blind) writes to the rest of the "4 GB", but then things go horribly wrong trying to read data back
 from the card.

ECC read

a significant amount of time to recover errors ... because it needs to process the original data as a whole

Yes, Hamming codes needs to process the "whole block", but for SD cards the "whole block" is only 512 + 16 bytes.

When your computer or microcontroller sends the "read a 512 byte block" to the SD/MMC card,
the SD/MMC card controller reads the full 512
user bytes + 16 ECC bytes, corrects the errors (if there were any), and sends the corrected 512 bytes of user data to your computer or
microcontroller
(or sends an error code if there were so many errors in the user data or in the ECC data or both that they were uncorrectable).

Because the on-card controller can read 512 + 16 bytes from the flash and do Hamming correction much faster than it trickles response codes
and data through the card interface, this is not noticeably slower than a (hypothetical) card without ECC.

As far as I know, SD/MMC cards don't use Reed-Solomon error correction -- but I suppose they could do that internally, and there's no way to tell
from the outside.

Share Cite Improve this answer Follow edited Apr 13 '17 at 12:32 answered Jul 10 '12 at 18:50
Community ♦ davidcary
1 16.7k 9 60 112

I do believe that SD card ecc is up to the manufacturer. They could use simple Hamming, Reed-Solomon or BCH (useful as capacities increase).
– tuxfool
Jul 16 '12
at 12:05

The ECC controller is usually embedded inside the MCU on the SD Card. The ECC controller is used to create ECC codes that get stored in the
spare area of flash (Each page in the flash contains spare capacity that can be used by the mcu).
4
It is also used to to compare the codes to the data being read (for error correction and detection). So each time a page is read the code is used to
correct data from that page (corrected data is not rewritten). It represents an overhead but if you look at it page by page then doing the
comparison on the fly is not particularly difficult.

so...

Write:
HOST(data)->MCU(writes ecc)-> FLASH(data+ecc)

Read:
FLASH(data+ecc)->MCU(reads ecc)-> HOST(corrected data)

Share Cite Improve this answer Follow answered Jul 10 '12 at 11:01
tuxfool
76 2

Erasure codes take a set of (k ) symbols and turn it into a larger set (n symbols) with redundant coding, so n > k. You stored (way) more
information than you would need to. That is where the redundancy comes into play. Think of it like taking a book, tearing out the pages, make
1 copies of them and distribute (random) sets of them among your friends. All the chunks that come out of the process usually have some sort of
integrity check (e.g. parity checks). If you read it, you select enough chunks to recover the k symbols and do the math. If the integrity check fails,
you replace that bad chunk with a new/similar one, or you reconstruct the missing information (depending on the code used).
A quite popular
code is the Reed-Solomon Code (pdf), so maybe that can help you as a concrete example.

To answer your related questions: The data is (inherently) verified when read (not when written) since it is cheaper. The writing is done to different
locations in parallel, so there is no extra delay from the outside perspective.

Share Cite Improve this answer Follow answered Jul 10 '12 at 8:58
suha
1,754 13 16

I'm familar with Reed-Solomon, it's also used in the PAR2 format for example. However, one property of it is that it takes a significant amount of time to recover
errors when you have only 10% recovery blocks, because it needs to process the original data as a whole to be able to recover just a small part. Also, does this
mean you cannot use 10% of the available space on the card?
–  Muis
Jul 10 '12 at 9:25

Surely there will be better introductions to RS than that 144 page scanned document.
– stevenvh
Jul 10 '12 at 9:31