Jörgs Digital Imaging Page

Eye (2k)
Why do two colours, put one next to the other, sing? Can one really explain this? (Pablo Picasso)

This page is a collection of tips and links about scanning, digital photography, image conversion and related topics - in short, "digital imaging".

General Information: Image resolution | Filesize
Image treatment: Storage format | Format conversion | HTML | Colour correction | Red Eye correction
Imaging: Scanner as camera | Archival with date-time stamp from Digital Camera
Presenting the results: Make a visual Directory ("Album") | Adding annotations | CD autoload
Miscellaneous: Scanning in Linux | What SCSI card? | Links

Thoughts about file size and resolution

How big is the file?

Almost all common image storage formats are more or less compressed. The size of the "uncompressed" image (in memory) can be calculated by considering size and colour depth. An example:

Another example:

If you scan lots of images (especially slides), you may want to make sure to have an appropriate archive medium at hand. I archive my scanned images on CD-ROM.

What resolution?

Generally speaking, you can always downsample a high-resolution image, but it is difficult to enhance a low-resolution file.

Digital photography: Usually, I take pictures at the full resolution of the camera and store the images in JPEG format. If available, I use a quality setting of "high" for small cameras and "medium" or "high" with high-end cameras. In exceptional cases (like studio photography), I may even use the RAW format.

Scanning photos: For archive purposes and when the object is of sufficient quality (slides, film negatives), I use a slide scanner and archive at 1600 or 2400 dpi. This is pretty much "borderline" since the film grain will start to be visible: The Kodachrome 64 film, famous for its fine grain, was specified with a resolution of 100 lines/mm under good conditions (contrast 1000:1). This resolution translates to 2540 dpi. With a contrast of only 1.6:1, the same film is specified at 63 lines/mm or "only" 1600 dpi.

However, if you need to scan low-cost prints, resolutions higher than 150 dpi will probably not make much sense.

For publishing on the web, I scan at a resolution that gives the final image size, or at a higher resolution followed by downsizing. Example: a photo ("print") of 10×15 cm that shall give an image of 400×600 pixels. In this case, with 10 cm = 4 inch, the required resolution calculates as 400 pixels/4 inch, or 100 dpi.

Image storage and conversion

What storage format?

You may want to store graphics in a format that is lossless and suited for this purpose.

There are many storage formats available. One of the most widely known is the JPEG file format, which is used by most digital cameras. However, JPEG was specifically designed for continuous-tone images such as photographs - it is clearly unsuitable for graphics, computer screenshots etc.

I store almost all of my "non-photograph" raster images as PNG files. The PNG (Portable Network Graphics) format is a file format for the lossless, portable, well-compressed storage of raster images and it is free from patent issues. PNG are still rather large files (roughly 25% of the calculated file size), but I feel that it is preferable to store high-quality images as "master copy". Later on, they can be converted to any "lighter" format as needed.

Image conversion

Conversion between different image formats, even (to a certain degree) between vector and pixel graphics, is easily done with the ImageMagick package. It is part of almost any Linux distribution and allows to display, convert, annotate, animate, transform ... images.

Let us convert a number of BMP files into PNG, using bash:

for i in *.bmp ; do convert "$i" "${i%.*}.png" ; done

If you have lots of files to convert, you may want to use parallel processing (source and more ideas):

ls -1 *bmp | parallel convert '{}' '{.}.png'

Convert PNG to JPG and create both thumbnails (150×100 pixels) and the "final" format (750×500 pixels), using bash:

for i in *.png; do 
  fn=${i%.*}
  convert -verbose -comment "(C) Joerg Hau" -geometry 750x500 $i ${fn}.jpg; 
  convert -verbose -comment "(C) Joerg Hau" -geometry 150x100 -quality 35 $i ${fn}_i.jpg; done
done

Note the low quality factor (0.35) for the thumbnail images - the smaller they are, the faster the page loads. If you put images on a web page, you want them to be "as light as possible".

Resize to fixed height

Panorama photos are great souvenirs. A technical challenge is that these photos are very large - almost unusable in a web browser. We cannot simply use the resizing above, since this would reduce the image size too much ... but ImageMagick provides a solution: If we specify only a fixed height (here, 900 px) but not the width; the image size will then be adjusted accordingly. In bash:

for i in *pano*; do convert -verbose $i -resize 'x900' ../img/${i}; done

Create HTML code

To create the necessary HTML hyperlinks for all images and thumbnails created above, issue the following lines in a bash shell and paste the output into your favourite HTML editor:

for i in *.png ; do
  fn=${i%.*}
  echo '<a href="'${fn}.jpg'" ><img src="'${fn}_i.jpg'" width="150" height="100" alt="'${fn}'"></a>'
done

Note that this assumes "all-landscape" orientation, so you have to "flip" the orientation of some images manually in the final HTML document. Further down this page you can find a script for this task.

Colour correction

Sometimes the scanned images present a "false" colour, or are too dark. I encounter this problem frequently when I scan slides that were exposed in bad weather - the result is often blueish and too dark (Digital cameras seem to excel under such conditions ;-)

Of course you may correct this by hand on a slide-per-slide base. However, often you can apply the same correction step to all slides in a given set - which calls again for a few lines in a Linux shell (here, tcsh). The following lines convert slides from .tif to .jpg and at the same time the gamma value for the red, blue and green channel is increased separately (1.3/1.3/1.2):

foreach i ( *.tif )
  set fn=$i:r
  convert -gamma 1.3/1.3/1.2 $i ${fn}.jpg
end

One example is shown here. First the slide "as acquired" and then the image that resulted from conversion with -gamma 1.7/1.7/1.6 (a somewhat extreme case):

Image as acquired After gamma correction

Red Eye correction

Especially with compact cameras, I frequently encounter images where people have "red eyes". This is usually caused by the electronic flash which is located too close to the axis of the camera.

Using The GIMP, this effect can be corrected as follows:

  1. Zoom onto a red eye.
  2. Use the rectangle or ellipse tool to mark the eye, including pupilla and iris.
  3. Context menu | Select | Select by Color.
    • Select enough "Fuzziness" (ca. 50).
    • Selection mode on "intersect".
    • Now mark an "average red" pixel.
  4. Verify that the selection is good enough (red button in the lower left corner). Correct if necessary.
  5. Context menu | Image | Colors | Hue-Saturation.
  6. Select "Magenta" and "Red" and reduce the values for Lightness and Saturation.

(Many thanks to Tibor D. from ch. comp!)

There is also a detailed article from Eric Jeschke on red eye correction with GIMP, which also provides a GIMP plug-in.

Digital Imaging

Using the scanner as a camera

A flatbed scanner can be used to take "photos" of small (and not-so-small) objects. For this purpose, the most important feature of the scanner is the depth of the focal plane - some scanners can only make images that are a few mm in depth, others support objects of 1...2 cm quite well.

To give a few examples, here is an extract of my wallet (left) and the diode plate of my motorcycle (right). In both cases, I just put the object(s) on the - clean - glass plate of the scanner and started the scan. Annotations were added later.

Swiss Francs Diode
 Plate

Note that my old flatbed scanner, a HP Scanjet 5P (C5110A), falsified the colors of metallic objects a bit. This is probably due to polarisation effects.

Archival from digital camera

Most digital cameras store photos with the date/time stamp when the picture was taken. I use this to archive my files based on their date, which avoids dealing with filenames like dsc0211.jpg.

The script below searches for a string that looks like a date and renames the file based on ISO8601 notation, i.e. yyyyMMddThhmmss. Potential duplicates - that is, files taken at the same second (!) - are renamed as file.jpg, filea.jpg, fileaa.jpg ... you get the idea.

bullet Click here to download the rename_photos.sh script (updated 2011-01-05).

Adjusting the date/time stamp as such is much more tricky, since you need to observe a number of caveats about rollovers. For this purpose, I use exiftool, which is part of the Perl module Image::ExifTool (in Fedora: package perl-Image-ExifTool). The following example shifts the creation date/time of an image by subtracting one hour and 11 minutes; note that the original file is modified:

exiftool -CreateDate-='0:0:0 1:11:0' img.jpg

Another example: adjust the three tags CreateDate, Date/Time Original and ModifyDate at the same time, by adding one hour. This is what I will use when I forget to set my camera to daylight saving time:

exiftool -AllDates+='0:0:0 1:0:0' img.jpg

Just for the record, renaming of files according to their last modification time can also be accomplished using a built-in feature of the find command. Here is an example with video files:

find . -type f -name "*avi" -printf "mv %f %TY%Tm%TdT%TH%TM%TS.avi\n"|sh

Another example, without the seconds and attaching the original filename after the DateTime stamp:

find . -type f -name "*MPG" -printf "mv %f %TY%Tm%TdT%TH%TM-%f\n"|sh

Presenting the results

Making an album

To present digital pictures, I usually put them in the format of an HTML page so that it can be visualised on any computer, and using any web browser. The page generation is largely automated by a shell cript that I called makealbum.sh. It supports:

The actual image conversion process is separated from the "user input" process, so you do not need to wait in between :-) All files are transformed on-the-fly into the parent directory, with automatic generation of thumbnails and of valid HTML code.

bullet Download the makealbum.sh script. Updated 2009-01-08.

bullet Download the dropshadow.pl script, required to add the DropShadow effect on-the-fly. This is a barely modifed script by Erich Lerch that uses the Gimp-Perl plugin. Updated 2005-03-08.

A related script is makealbum_name_tags.pl. It is a simple perl script that replaces text strings in one file based on a list in another file (see the file itself for the details). I use it to replace the date-time stamp (in the "name" tags of the HTML files generated by makealbum.sh) with textual descriptions.

bullet Download the makealbum_name_tags.pl script.

Yet another ;-) related script is makealbum-multipages.sh. It is a bash shell script to create multiple HTML files that reference each other - something that I needed for multi-page trip reports in HTML format (e.g. one HTML page per day). In practice, I set up the "framework" with makealbum-multipage.sh, then do the image conversion/rotation etc. with makealbum.sh and copy the corresponding sections from makealbum.sh's output into the files created by makealbum-multipages.sh.

bullet Download the makealbum-multipages.sh script. Updated 2008-01-25.

You can see some pages prepared with this script e.g. on my motorcycle pages.

Annotating Images

This is actually a complement to the scripts above. img_annotate.sh allows to add text under a jpeg image from the command line. I use this frequently to add comments, descriptions or names into photos.

The script requires the name of the image to be treated, plus the text ... e.g. img_annotate.sh 01.jpg "My comment is here". The resulting file is stored with a different name (here: 01t.jpg ... "t" for "text" ;-), so the original file is never modified.

bullet Download the img_annotate.sh script.

CD autostart

If you burn your image albums onto CD and would like to start a "slide show" automatically, there are numerous programs available that perform this purpose. However, a pretty easy trick that I frequently use for CDs with images (presented as "HTML picture album") is the following:

  1. Create a file index.html that contains the navigation through your CD.
  2. Create a file autorun.inf with the following content:
    [autorun]
    open=autorun.bat
    
  3. Create a file autorun.bat with the following content:
    @echo Launching "index.html"...
    @start index.html
    @exit
    
  4. Make sure the files are "DOS type" (i.e. using CR+LF at the end of a line), otherwise they may not work in MS-Windows 9.x.
  5. Put all three files in the root directory of your CD, then burn the CD.

If you insert this CD into the CD drive of an average MS-Windows system, the CD will launch your default web browser with the presentation.

Scanning in Linux

Scanning in Linux .... and over the Network

Using scanners under Linux is easy: Get the sane package. Sane stands for "Scanner Access Now Easy". For a graphical interface I recommend to use XSane, which has grown into an amazing piece software - it is not just a scanner frontend, but allows e.g. scanning to different formats (including PDF) and offers an interface to the OCR software gocr. Some programs, like GIMP, allow to scan from inside the application; use the menu File | Acquire.

Even better: Linux allows also to use your scanner over the network! The required program is the "SANE daemon" (saned) and is part of the SANE distribution.

To configure a SANE client (that is, the PC that is supposed to receive the scan), edit /etc/sane.d/net.conf and add the name (or IP address) of the SANE server. That's all.

To configure the SANE server (that is, the machine where the scanner is physically attached to), proceed as follows:

Scanning to PDF

If you are interested in scanning documents to PDF, please refer to my documents page.

The Importance of the SCSI Hostadapter

Note: This section was written at a time when USB did barely exist and most scanners needed an interface card. In the 21st century, you will most probably use a USB scanner.

Does your scanner "block" the PC while you are scanning?

This is not the "fault" of the scanner - it is the interface card. There are several types of SCSI hostadapters ("SCSI cards") on the market. Those usually delivered with low-cost scanners are generally working in PIO mode, which means that the processor of the PC takes care about all the data transfer between your scanner and the harddisk. Expensive, "real" SCSI adapters have their own processor onboard and work in DMA mode, freeing the processor completely.

Here's data from a little experiment I performed several years ago:

The scanner used here was a Highscreen "Scanner 600", a low-cost but well-working flatbed scanner with 600 dpi optical resolution. The "object" scanned was an ordinary phonebook. The tests were run on a generic Pentium-II 233 MHz with 64 MB RAM and ABIT BE6 motherboard. The PC was running SuSE Linux 6.3 with Kernel 2.2.13, with no other applications running.

Scanning was performed from the command line: time scanimage --mode col --resolution 300 -x 160 -y 230 > test.png. All acquisitions were repeated three times; data were averaged (the standard deviation was neglible).

Results:

Conclusions:

The total time used for the scanning process does not depend on the type of SCSI adapter: it was identical in both cases. The big difference lies in the CPU time that is used for the scanning process!

The AVA-1502A is a low-cost adapter working in PIO mode. It keeps the CPU busy most of the time (60 seconds here). What this means? Well, the PC will be occupied with data transfer all the time. It will have a lousy response to the user's input - the mouse feels "sticky" and under Win95 the PC is unusuable during the whole scanning process!

On the other hand, the AHA-1542 is an old, but "true" SCSI card operating in DMA mode. It frees the computer's CPU completely from workload - it spent only 80 ms (milliseconds!) of time on the process! Sitting in front of the machine, you will not be able to say if the PC is doing something on the SCSI chain or not. The PC behaves "as normally" and the scanning process does not influence usability.

As a consequence, you may want to get a SCSI adapter that supports Bus Master DMA and you may want to stay away from the cards that support only PIO modes. They are cheaper and you will feel the difference.

Links

Links around Imaging and Scanning