====== Negative scan with GNU/Linux and Canon CanoScan 9000F Mark II ======

See also the page **[[canoscan_9000f_mark_ii]]**.

===== The first negative scan =====

I purchased the scanner mainly to acquire my old negatives, so I put two strips of Fuji negatives into the rails and I did my first test.

After starting the **xsane** program, from the preview window, I selected the area of negatives rails (about 70 x 240 mm). You can add a preset area using the context menu, which add the following section into **''$HOME/.sane/xsane/xsane.rc''**:

<file>
"preset-area-name"
"CanoScan 9000F - Negative rails"
"preset-area-xoffset"
4795735
"preset-area-yoffset"
423738
"preset-area-width"
4587520
"preset-area-height"
15728640
</file>

{{  .:canon-900f:canon-9000f-mark-ii_fuji-negative.jpg?direct&250|Negative scan at 2400 dpi}}

Then I selected //Transparency Unit// as the scan source, //Color// as the scan mode, //Fuji// as the medium type (that film was a Fuji), 2400 dpi, clicked the //Negative button//, clicked the //Set default enhancment values// (shortcut Ctrl-0) to reset gamma controls to their central position. The acquire process tooks about **3.5 minutes** (206 seconds), and produced a 16 bit/channel TIFF file of 650 Mb. I used a very low-end desktop with 2 Gb of RAM and and Atom CPU D510 at 1.6 GHz.

A standard 135 mm negative acquired at **2400 dpi** produces an image of about 3400 x 2200 pixels (i.e. about **7 Mpixels**), the result is very light, as you can see. The scanner is capable up to 9600 dpi.

===== Xsane and medium definition =====

See the Xsane documentation about **[[http://www.xsane.org/doc/sane-xsane-medium-definition-doc.html|medium definition]]**.

With the Xsane program it is possible to set varius levels for gamma correction, luminosity and contrasts or choose from some presets, including some brand of negative films. It is also possible to define new profiles, e.g. to get best results with this scanner and some type of negatives.

Neverthless it seems that none of that controls influence the way the scanner acquire from the transparency unit, so it is preferible to acquire without any correction (sort of //raw// scanning), and do postprocessing later, with some software.

===== PIXMA backend options for transparencies =====

Sane PIXMA backend allows several options to be passed using **scanimage**, you can see all of them with:

<code>
scanimage --device-name pixma:04A9190D --all-options
</code>

this is a summary (run the command to get some extra explanation):

<code>
  Scan mode:
    --resolution auto||75|150|300|600|1200|2400|4800dpi [75]
    --mode auto|Color|Gray|48 bits color|16 bits gray|Lineart [Color]
    --source Flatbed|Transparency Unit [Flatbed]
    --button-controlled[=(yes|no)] [no]
  Gamma:
    --custom-gamma[=(auto|yes|no)] [yes]
    --gamma-table auto|0..255,...
    --gamma auto|0.299988..5 [2.2]
  Geometry:
    -l auto|0..216.069mm [0]
    -t auto|0..297.011mm [0]
    -x auto|0..216.069mm [216.069]
    -y auto|0..297.011mm [297.011]
  Buttons:
    --button-update
    --button-1 <int> [0] [read-only]
    --button-2 <int> [0] [read-only]
    --original <int> [0] [read-only]
    --target <int> [0] [read-only]
    --scan-resolution <int> [0] [read-only]
  Extras:
    --threshold auto|0..100% (in steps of 1) [inactive]
    --threshold-curve auto|0..127 (in steps of 1) [inactive]
</code>

**WARNING:** The order of parameters does matter! For example if you select **''%%--source "Transparency Unit"%%''**, the **''%%--resolution%%''** parameters will be resetted to the default value of 300, so **you must set the resolution after the source**! Tip: run the program with the options you want and append the **''%%--all-options%%''**: you will see all the actual and available options.

I was mainly interested into **gamma** and **color correction**, which is the real problem when scanning negative films. I played with all the gamma options, but I get always the same image as result. After being confirmed by this post [[http://en.it-usenet.org/thread/12840/2094/|pixma: no gamma table in TPU mode]], I'm rather sure that this scanner **does not support gamma when scanning transparencies** (Transparency Profile Unit).

We can assume that this scanner (at least with libsane 1.0.24) does a **raw scan** of transparencies, which is a good thing in my opinion. At scan time we have not to bother with gamma correction and color calibration (e.g. using different profiles with different brand of negatives) and we can proceed in quick batch. Then we will do **all the postprocessing via software**, keeping the original raw data for future enhancements.

===== Scanning from command line and benchmarking =====

**WARNING**! The order of parameters passed to ''scanimage'' does matter! See above.

**WARNING**! With images acquired at 4800 dpi, **64 bit software** is probably required (Imagemagick, tiffinfo, etc.). Otherwise integer overflow may occurr.

With this command we will scan the entire transparency surface, containing **two strips of 135 mm negatives**:

<code>
scanimage --device-name pixma:04A9190D \
    --source 'Transparency Unit' \
    --resolution 2400 \
    --format tiff \
    --mode Color \
    -l 76 -x 66 \
    > negative.tiff
</code>

In the following table we report the scan time and the size of the resulting file, using different resolutions:

^  Resolution  ^  Scan time  ^  Scan size  ^  File size (tiff)  ^  Single shot size  ^  Mpixel  ^
^  1200 |  0m 48s |   3118 x 11840 |  212M |  1692 x 1128 |  ~  2 | 
^  2400 |  2m 39s |   6236 x 23680 |  845M |  3384 x 2256 |  ~  7 |
^  4800 |  5m 54s |  12473 x 47360 |  3.4G |  6768 x 4512 |  ~ 29 |
===== Infrared =====

{{.:canon-900f:canon-9000f-mark-ii_infrared.jpg?direct&100 |Infrared scan of negative film}}

The CanoScan 9000F Mark II scanner has an infrared sensor, useful to remove scratches and dust from the scanned image. With xsane it is possible to acquire the infrared image, just select //Infrared// as scan mode. The result is a grayscale image, where only dust and scratches are visibles. This image should be used togeter with the color scan, to obtain an interpolated image. FIXME How to do that?

===== Batch processing =====

==== Acquire ====

The scanner accomodates two stripes of 135 mm negatives. This is the script used to acquire them:

<code bash>
#!/bin/sh
filename="$(date +%Y-%m-%d_%H%M%S)"
resolution='2400'
format='tiff'
scanimage --device-name pixma:04A9190D \
    --source 'Transparency Unit' \
    --resolution "$resolution" \
    --format "$format" \
    --mode Color \
    -l 76 -x 66 \
    > "${filename}-${resolution}.$format"
</code>
==== Strip cut and rotate ====

Then we used this script to cut the two stripes and rotate them by 270 degrees. The png compression seems to be the best lossless method:

<code bash>
#!/bin/bash
# Cut two strips from a 135 mm negative scan, CanoScan 9000F Mark II

filename="$1"
resolution='2400'
format='png'

case "$resolution" in
    1200)
        strip1='1250x10850+0+925'
        strip2='1250x10850+1855+925'
        ;;
    2400)
        strip1='2500x21700+0+1850'
        strip2='2500x21700+3710+1850'
        ;;
    4800)
        strip1='5000x43400+0+3700'
        strip2='5000x43400+7420+3700'
        ;;
    *)
        echo "Usage: $(basename $0) file.tiff"
        exit 1
        ;;
esac

# To avoid "the PNG file specifies an offset" problem, use the +repage option.
set -x
convert "${filename}" \
    -crop "$strip1" \
    -rotate 270 \
    +repage \
    "${filename}_A.$format"

convert "${filename}" \
    -crop "$strip2" \
    -rotate 270 \
    +repage \
    "${filename}_B.$format"
</code>
==== Invert and color balance ====

This is an empirical recipe to invert and balance colors from a scanned negative strip. It is based on a three pass process: **negate**, **gamma correction** and **color level adjust**. The values for gamma correction and color adjust, make a specific profile suitable for just one brand and type of negative (e.g. Kodak VR 100-3).

Scanner softwares often contain several negative profiles based only on color adjust. Gamma correction is defined for each RGB channel along the black and white point. We instead prefer to apply an uniform gamma correction before color level adjust, so we can deal with under-exposed or over-exposed frames just touching a single pre-process parameter.

First of all we make several test from a negative scan, setting different gamma correction:

<code bash>
!/bin/sh -x
test_values="0.002 0.003 0.004 0.005 0.006 0.007 0.008"
for gamma in $test_values; do
  convert -colorspace RGB "$1" \
    -negate -gamma $gamma \
    "P_g${gamma}_$1"
done
</code>

We choose by visual inspection the best value for gamma; 0.007 in this example.

Thake a film strip (negated and gamma-corrected with the above recipe) and open it with the Gimp. We need a strip with good **black**, **gray** and **white** spots. The process is:

  - Apply a **pixelize blur filter** on the black, gray and white spots. Make it large enough so you can easly pick the color from it in the next steps.
  - Open menu **//Colors//**, **//Levels//**:
    - From **//All Channels//**, click **//Pick gray point//** and then click on the gray spot.
    - From **//Channel//** select **//Red//**, click **//Pick black point//** and click on the black spot. Click **//Pick white point//** and click on the white spot.
    - Repeat the above step for **//Red//**, **//Green//** and **//Blue//**.
  - Before closing the dialog box, annotate the values of **black point**, **gamma**, and **white point** for each RGB channel.

Suppose that the values are:

^ Channel  ^ Black  ^ Gamma  ^ White  ^
| Red      |     22 |   1.06 |    193 |
| Green    |     20 |   1.01 |    190 |
| Blue     |     30 |   0.78 |    197 |

The black and white points are integers 0-255, we need to convert it into a percent value; e.g. for the red white point: 22 / 255 * 100 = 8.63%.

This is the Imagemagick **''convert''** recipe:

<code bash>
!/bin/sh
for file in $@; do
  convert -colorspace RGB "$file" \
    -negate \
    -gamma 0.007 \
    -channel R -level 08.63%,75.69%,1.06 \
    -channel G -level 07.84%,74.51%,1.01 \
    -channel B -level 11.76%,77.25%,0.78 \
    "P_$file"
done
</code>
==== Cutting frames using the GIMP ====

Now we have several strips of frames, acquired with batch procedures and very few manual work. The last step indeed requires some manual work and human intervention: **cut the single frames**, eventually **rotate** them, and **save** in our preferred format.

We use the **[[http://www.gimp.org/|GIMP]]** with a **preset** for the //Rectangle select tool// and a **plugin script** named //Export selection as jpeg//. Prepare the GIMP:

  - Save the file **''{{.:canon-900f:135-mm-frame-select-2400.gtp.txt|135-mm-frame-select-2400.gtp}}''** into your **''$HOME/.gimp-2.8/tool-presets/''** directory.
  - Save the file **''{{.:canon-900f:export-selection.py|export-selection.py}}''** into your **''$HOME/.gimp-2.8/plug-ins/''** directory.
  - Assign a keyboard shortcut the the //Export selections as jpeg// plugin, eg. **''Ctrl-Alt-E''** (menu //Edit// -> //Preferences// -> //Interface// -> //Configure Keyboard Shortcuts// -> //Search Export selection...//).

Now the workflow will be:

  - Open the strip image
  - Press **R** to activate the //Rectangle select tool//.
  - From the //Tool Options// dialog, click //Restore Tool Preset// and choose "135 mm frame select @2400" (there is a bug? You may need to choose it two times): you see the **fixed size** for selection.
  - Select the frame you want to save (the preset will force you with the fixed size).
  - Press **''Ctrl-Alt-E''** to run the script: insert the filename and the required rotation.
  - Repeat the last two steps.


===== ImageMagick convert, memory issue =====

Executing **convert** on very large images can lead to memory exhausted issue. I experienced it converting a TIF scan into PNG (because I wish to avoid 24 bit Gimp limitation with TIF images):

<code>
convert very-big.tif very-big.png
convert-im6.q16: DistributedPixelCache '127.0.0.1' @ error/distribute-cache.c/ConnectPixelCacheServer/244.
convert-im6.q16: cache resources exhausted `very-big.png' @ error/cache.c/OpenPixelCache/3945.
convert-im6.q16: No IDATs written into file `very-big.png' @ error/png.c/MagickPNGErrorHandler/1628.
</code>

It seems that ImageMagick is self-limiting the disk resource it will use, you can fix the configuration file **''/etc/ImageMagick-6/policy.xml''**, increasing the **1GiB** limit:

<code xml>
<policymap>
  <policy domain="resource" name="memory" value="256MiB"/>
  <policy domain="resource" name="map" value="512MiB"/>
  <policy domain="resource" name="width" value="16KP"/>
  <policy domain="resource" name="height" value="16KP"/>
  <policy domain="resource" name="area" value="128MB"/>
  <policy domain="resource" name="disk" value="1GiB"/>
</policymap>
</code>

May be you can also increase the Memory resource, if you have enough:

<code>
convert -list resource
Resource limits:
  Width: 16KP
  Height: 16KP
  Area: 128MP
  Memory: 256MiB
  Map: 512MiB
  Disk: 1GiB
  File: 768
  Thread: 2
  Throttle: 0
  Time: unlimited
</code>

===== Web references =====

  * [[http://125px.com/articles/photography/digital/invertingraw/|Inverting raw scans]]
  * [[http://125px.com/articles/photography/digital/colorneg/|Thoughts on Color Negative Scanning]]
  * [[https://sites.google.com/site/negfix/|Negfix8 scrip]] to turn scanned negative image into the positive image.
  * [[https://zephyrsoft.org/blog/34-digitize-slides-and-prints-on-linux|Use Canon CanoScan 9000F Mark II to digitize slides and prints on Linux]], with some useful tips about doing crop and trim with Imagemagick, etc.