Photo scanning tips
The goal when scanning should be to capture all the detail in the original image. Scanning at 300 dpi is not sufficient, as a general rule, although it may be just fine when scanning an enlargement. The technical requirement (Shannon's sampling theorem) is that the sampling rate be at least twice the highest spatial frequency (the Nyquist frequency) to avoid loss of information and prevent introducing artifacts. Scanning at 600 dpi is barely adequate to capture the detail in 19th century prints such as stereoviews or wood block engravings. Scanning at 300 dpi (at 100% of the original) will also not provide sufficient detail when printing even a moderately enlarged copy from a small picture (such as would be desired for printing a copy of a detail from a stereoview or CDV in a textbook), and would provide disappointing quality if used for an exhibit display print. Scanning engravings at 300 dpi will result in severe Moire pattern artifacts.
TIFF images may be uncompressed, but TIFF allows a variety of compression schemes. Some are lossless such as LZW (based on substring encoding), others such as jpeg (based on Fourier transforms and cosine waves) are lossy (yes a TIFF image can use jpeg compression).
Having multiple backups is not sufficient – to survive long term they must be in different locations that are not subject to a common disaster such as a fire, flood, earthquake, war, etc. Also, each time that a backup format (whether the file format, the physical medium, the electrical interface, the computer instruction set, or the operating system, etc.) is superseded by a new one, you likely will need to make lossless copies into the new format or lose the pictures. There is a high risk that even if the physical medium, such as a disk survives with the data intact, it may be impossible or impractical to find a working device that can read the long obsolete format. How would you go about reading a well preserved DECtape or 8" floppy disk, for example, now ... in a hundred years? Where would you plug in your SCSI drive or RS-232 cable, now that your computer doesn't use such interfaces, and how would you obtain software drivers that work with the current operating system? Will your great grandchildren even know that some unfamiliar object contains pictures, or remember the password? ... Will they realize that the postage stamp size SD flash memory card that they are about to toss in the trash has a thousand pictures on it?
posted from CPRR Discussion Group at 10:32 AM
21 Comments:
From: Clifford Vander Yacht
Sally Jacobs does a fine job except in one area, #4 Saving in the wrong file format.
She says "For each scan, save an archival master as a TIFF file." This is good advice but it misses the bigger problem in between scanning or photographing and storage. The better statement is "Never save a file in JPEG format." Although that has become a standard format, especially of railroad photographers and is the internal format of many cameras, it was selected on the basis of faulty information. The internet wants either JPG or GIF formats. GIF is fine for black and white photos and line drawings and it is a LZW compressed file (the same compression as TIFF when it is compressed with that option). It is lossless and has 256 levels of brightness. For color, it appeared that GIF was bad as it caused rainbows (color steps, not red green and blue stripes) in the sky of otherwise nice photos. JPG didn't do that. What it does is to compress a 16 pixel square with wonderful 24 bit (16 million colors) color, but only for the average color. You can get monstrous blocks of sky exactly the same color. You also get what I call splatters along any sharp edge. If the photo isn't compressed much, such as in your camera, the result is still pleasing in a 5 megapixel image or larger.
You may make COPIES of JPG files. Don't use an image program such as Adobe PhotoShop, but use Windows Explorer, the program that came with Windows. Find it. If your camera is plugged in and turned on, you should see the camera as an entry in "Computer", just like your C: drive, etc. On my computer, it shows up as drive F:. Click on "Memory Card (or Internal Memory if that's where your photos are), DCIM (or whatever your card has), then something like "100K7530" and Windows XP will show you thumbnails of the contents of the memory card in your camera. You can now copy whatever selection you want to your computer. They will be copied in the original format. You can archive that as is. That is your photo except it is now in your computer (and in your camera unless you deleted it while there -- yes, your computer can read and write your camera).
When you access those JPG files, never ever, save them in JPG again! Use BMP (uncompressed and a very old standard file format which your computer uses for your desktop background), or use TIF (the extension) of TIFF(Tagged Image File Format) in either uncompressed, LZW compressed or the current favorite, RAW.
For printing, the digital image must be 400 ppi or more for any quality printing. That's 400 pixels per inch for every inch of the final print size. Anything less will have obvious small blocks called pixelation. To get the proper size file, you probably may increase the size using PhotoShop by changing image size to 200%.
If you have to scan a previously printed black and white photos, see my explanation of a method to actually improve the detail of the image.
—Cliff
Using only 256 levels (8 bits) per pixel per color (or gray in the case of B&W) may not be adequate to preserve tonal information -- much safer is 16 bits per channel. (There is a very old myth that people can see less that 256 shades of gray, but this is wrong; it depends on the image and viewing circumstances. Even when 256 shades is adequate for the original image as taken, it may be insufficient to support subsequent digital correction of the tonal range.)
RAW is not an archival format, is not standardized, nor necessarily publicly documented, and should never be the only format saved! RAW saves all the information collected by each camera in the proprietary format for that device, so gives maximum flexibility for further processing. But because this format is likely to be different for each manufacturer and camera model, it is almost certain to be the among the first formats that will no longer be readable by new software as soon as the camera model is obsolete and no longer supported. The RAW format may even be protected by intellectual property laws and/or encrypted so that third party software is prevented and/or prohibited by the camera manufacturer from independently supporting their RAW format.
BMP is mostly a Windows format.
Another effective way to descreen is to scan at a sufficiently high resolution, such as 600 dpi, then apply a Gaussian filter adjusted to blur sufficiently to just remove the printing halftone screen dot pattern, then reduce the image size by resampling to the desired final image size, then sharpen. If original was an albumen print, we also convert from monochrome to RBG if needed, and then colorize the image to sepia by adjusting the Red, Blue, and Green levels, and the color saturation. Using this procedure we often receive a comment from a surprised editor that the images look better on our website than they did in their magazine.
From: Clifford J Vander Yacht
Whatever works. I tried the Gaussian blur and found it lost too much detail. I was first writing a program to calculate the effects of the dots and remove them. Because scanning and/or printing causes shifts in the pattern, it didn't work for more than a few printed lines. So I thought of the filter grid, which worked well. It is quite surprising to see details you didn't know were there.
You can't tell the source of images in The Prospector, the magazine of The Rio Grande Modeling & Historical Society now.
—Cliff
From: Clifford Vander Yacht
> Using only 256 levels (8 bits) per pixel per color (or gray in the case
> of B&W) may not be adequate to preserve tonal information
It is the current standard. It works for printing which is the major use of photos. Unless RAW or some other format become the defacto standard, then the technology may just fade with time.
> BMP is mostly a Windows format.
Ah, the Mac/Windows war surfaces. However, the BMP format is simple, it can even be deduced from looking at the file structure in Debug (remember that? That's the language Bill Gates wrote for IBM). And there are many books stashed away which contain the information – a very distributed archive.
I can still convert 5-1/4" floppies. New computers don't even have a floppy drive.
—Cliff
Certainly 256 levels (8 bits) is fine for most applications such as printing, but the cited article was about archiving for future use which is quite a different problem. What works for printing now may not work for multiple generations of future tonal modifications. Many people are not aware that scanner output exceeds 8 bits and that professional software such as Photoshop can save and work on 16 bit/channel images. Scanning a negative and converting the scanner output to 8 bits to save space (which is the usual default) results in throwing away unrecoverable tonal details. This is what is typically done, but don't think that when the original has deteriorated or is no longer available to rescan that the 8 bit "archival" copy has faithfully preserved the image.
The point about BMP is that if you chose to archive images using an obsolete, now somewhat obscure, platform specific format, you increase the likelihood of there being future difficulty in reading the images. It is good that the BMP format is documented, but as a practical matter, if future software generally doesn't support an obsolete format, few people will bother to find or be able to program a converter.
From: "Sally J. Jacobs, Archivist" sally@jacobsarchival.com
Thank you so much for your careful review.
One change I will make in the next edition is to remind folks that it is a quick guide – a way to become acquainted with the challenges of digitization. I will also point folks to more in-depth resources. What would you recommend? (Keeping in mind that many of the readers will be such newbies that the idea of an uncompressed digital master will be brand new to them...)
I agree 100% about the long term fragility of digital records. Are you familiar with the Long Now Foundation? There is an essay I love by Stewart Brand, it's called "Written on the Wind."
–Sally
----------------------------------------
Sally J. Jacobs, Archivist
Email: sally@jacobsarchival.com
Phone: (608) 332-1494
Web: http://www.jacobsarchival.com
----------------------------------------
JACOBS ARCHIVAL SERVICES
Our business is saving memories.
+ Scanning & Digital Restoration
+ Family History & Tribute Books
+ Organizing & Clutter Control
+ Long Term Storage Solutions
Here are a number of more in-depth resources about digital image archiving:
Best way of preserving your digital content?
Library of Congress, Digital Preservation: The National Digital Information Infrastructure and Preservation Program
Digital Preservation: How Long will Digital Photography Last?
Digital Longevity: the lifespan of digital files
Information Longevity
Storing & Archiving Digital Photos
Labelling Digital Photos
Preservation and Archives Professionals
Digital Images Archiving Study
"There is only one flavour of TIFF suitable for long term preservation: un-compressed Baseline Revision 6. It should not use any compression and should not use any of the additional functionality available in some other revisions."
JPEG2000
Technical Guidelines for Digitizing Archival Materials for Electronic Access
Online Archive of California Technical Information
Digital Imaging: Imaging and imagebases
Archival Gold DVD's
Congratulations on your efforts to try to make this information accessible to a non-technical audience.
From: Clifford J Vander Yacht
You are correct and that does apply to those who wish to archive photos as a profession or an extensive hobby. However, the worse problem at this time is the typical railfan and hobby photographer who has no intention NOW of ever saving anything for the future (the NOW generation). Thus I will stand on my soapbox and shout, "NEVER IN JPG" at the top of my voice and tell him/her how to save it in anything else so that it can be printed in at least black ink. Or, at least don't destroy what you have.
For the internet, I use GIF as much as possible except for large color photos. Even then I compare file sizes. The quality doesn't need to be good. For GIF color, I reduce to 256 colors using a dither. See the [illustrations in the online R&LHS Newsletter.] That keeps the detail but makes color spots which nicely blend in the eye, but impossible to print. But as a professional photo printer, I found most people's idea of quality doesn't even come close to yours or mine.
—Cliff
GIF is excellent for images that are created with vector graphic software with filled in shapes of single colors, like cartoons or diagrams, when a total of no more than 256 different colors are used. However, it is a poor choice for continuous tone images both because the color subtlety may be lost while dither or contouring artifacts are typically introduced, and because continuous tone GIF images tend to be larger and/or of worse quality than corresponding JPEG images.
See the International Imaging Industry Association's discussion of photo archiving, Are Your Memories Safe?
See the Digital Photography Best Practices and Workflow Project by the the American Society of Media Photographers (ASMP) and the Library of Congress.
"A recent study from DTI/Price Waterhouse Coopers says that 7 out of 10 firms that experience major data loss go out of business within a year. And if you use a computer, there's a 25% chance of losing your data in any given year according to Gartner." —IEEE Spectrum
Also see, The Tao of Backup.
From: "Chris Cummins" chris@glowimagery.com
Subject: How To Preserve Old Photos
I was searching for some articles about archiving of family documents and photographs today and I came across this page on your site.
I noticed that you linked to one of my favorite articles about how to preserve old family photos in that article.
Just wanted to give you a heads up that I have created something similar. Mine is How To Preserve Old Photos Without Losing Your Mind. It’s like the article you mentioned but it is more focused on simplifying the overwhelming process of turning old family photos into an organized, safe and searchable digital archive with tips for how to preserve the film and paper originals.
... keep up the awesome work! ...
—Chris Cummins, Chief photographer/Owner Glow Imagery
Also see Thom Hogan's Data Recovery Interview with with David Zimmerman, CEO of LC Technology, Limited.
See, Digital Photo Restoration & Black and White Carbon Pigment Printing. Elverhoj Museum. By Paul Roark
Concerned about the possible longevity of "awesome" Helium filled disk drives because helium is so small and diffuses easily through materials. Are these helium drives only good for the duration of the warranty period?
A helium filled rubber balloon loses its helium in about 2-3 days. A helium filled metal coated mylar balloon loses its helium after about 2-3 months. How long will the required concentration of helium be maintained in these hard drives?
The longevity of archival magnetic hard drives is especially important now as we increasing use leaky electric charge on solid state drives (SSD's) as the primary file copy. The capacitance storage of SSD's, USB thumb drives, and camera cards likely will not endure long term. Nor will fading and UV sensitive organic dyes on writeable DVD's, etc. (Hopefully inorganic storage on M-discs will last, but who knows?)
So, what happens if the helium is lost? Does the drive degrade gracefully or damage the surface permanently, losing the data? Is there a helium recharge port, with helium canisters, hoses, etc., and instructions available? Will the ability to recharge helium be available in 5, 10, or 20 years should you need to retrieve old files? What about that archival hard drive lost in a storage box for a hundred years?
It is a good idea to keep hard drives forever as backups, in case you find out years later that there was bit rot introduced into later copies, so the only good copy is on a years old long obsolete and out-of-service archived hard drive.
Really upset with Apple for abandoning their ZFS project and then making it worse by not including data error detection/correction in their new file system.
Without thoughtful engineering, it is just too easy for our cherished digital photographs to become lost to our great grandchildren.
"My advice is to refresh archival files every 3-5 years on multiple duplicate copies with the latest and most robust storage." [MORE]
Great advice, but unrealistic. If all your archival media self destruct after 3-5 years, it is highly unlikely that any of your pictures will survive through multiple generations. Some of your descendants undoubtedly will put your digital storage in a cardboard box in the attic where it will sit forgotten for decades. Need to use digital storage that will survive that.
Also see,
Hollywood Loves Backups
How to Talk to Your Family About Backups
Post a Comment
<< Recent Messages