{
    "default_index": "eng_ocr",
    "indexes": [
      { <MyDifferenceBlockAlgorithm> },
      { <MyOCRAlgorithm> }
    ]
}

{
    "displayName": "Screen Diff Block",
    "indexer": { <IndexerPipeLine> },
    "name": "diff_block",
    "ocr": { <OCRPipeLine> }
}

Takes the image and create a single HSV color tuple (hue, staturation, value) by resizing the image to a single pixel. The HSV tuple is added to the meta data and the image itself is left unchanged.

{
    "action": "indexHSV",
    "options": {
        "tolerance": 0.05
    }
}

Takes the image and sharpens it.

{
    "action": "sharpen",
    "options": {
        "sharpness": 4.0
    }
}

Takes the image and crops it, based on the variable given. In most cases this will be the "$block" variable, that in the case of the indexing pipeline, will reference the translated textbox, while in the ocr pipeline, it will reference the shortlist element's textbox. For the most part, the variables referenced in the example below will be what you want.

{
    "action": "crop", 
    "options": {
        "color": "$block.colors", 
        "x1": "$block.bounding_box.x1", 
        "x2": "$block.bounding_box.x2", 
        "y1": "$block.bounding_box.y1", 
        "y2": "$block.bounding_box.y2"
    }
}

Used only in the indexer algorithm. Creates an index based on the image and meta data as currently processed.

{
    "action": "createIndex", 
    "options": {}
}

Used only in the ocr algorithm. Uses the previously defined index to find a shortlist of candiates to fetch.

{
    "action": "findShortlistByIndex", 
    "options": {}
}

Used only in the ocr algorithm. Expands out the shortlist list found previously into a list of images in the pipeline. Used later to check the image against against the various candiates found.

{
    "action": "expandShortlist", 
    "options": {}
}

Takes the image and does a difference between it and the image referenced with the variable name. If the images are close enough, then the meta data for this enty will be marked as a match, and have the original image use the translated textbox of the entry.

{
    "action": "diffImage", 
    "options": {
        "color": "$block.index_image"
    }
}

v1.01:Isolates blocks of "text" of the specified colors in the image without using OCR. See example pipelines for more information. .

{
    "action": "getTextBlocks", 
    "options": {
        "colors": ["FFFFFF"],
        "threshold": 32,
        "scale_factor": 10,
    }
}

Takes the image and reduces the colors to the ones specificed in the colors list. If a color in the image is not within the tolerance given, then it will use the color black ("000000"). The field "threshold" gives the tolerance to use, as an integer, with the equation:
(p.red-c.red)^2 + (p.blue-c.blue)^2 + (p.green-c.green)^2 < threshold^2
Where p is the pixel in the image being checked, and c is the color to reduce to. Color values themselves are integer bytes (0-255), so the threshold value should match this (for example: 32 is a good starting value).

Note that even if the color being selected from the image is an exact color (like "FFFFFF"), a tolerance of 1 may not work. As part of this step, the image input is first reduced to use a 256 color palette, and colors are then reduced based on the palette of this image. As a result, the reduction to 256 colors may change some colors on the screen to get a better "match" for the 256 color palette. If this is the case, then a higher threshold will help.

{
    "action": "reduceToColors", 
    "options": {
        "colors": [
            "FFFFFF", 
            "FFDF00", 
            "EFA20C"
        ], 
        "threshold": 32
     }
}

Similar to reduceToColors, but input colors can be mapped to different outputs. As well, a "base" field is added to specify what a non-matched color should be mapped to.

Changed in 1.01:Base can be null, which will leave non-matched colors alone. Color inputs/targets can be ranges, which will map colors between the end points to the corresponding two colors in the range that is closest. So for the range [["000000", "FFFFFF"], ["000000", "FF0000"]], the color "666666" will map to "000000", while "999999" would map to "FF0000".

{
    "action": "reduceToMultiColor", 
    "options": {
        "base": "FF0000", 
        "colors": [
            ["FFFFFF", "FFFFFF"], 
            ["FFDF00", "FFDF00"], 
            ["EFA20C", "EFA20C"], 
            ["000000", "000000"]
        ], 
        "threshold": 32
    }
}

v1.01:Expands target color in image over the background color by 1 pixel horizontally.

{
    "action": "expandColorHorizontal", 
    "options": {
        "target": "FFFFFF",
        "base": "FF0000"
    }
}

v1.01:Expands target color in image over the background color by 1 pixel vertically.

{
    "action": "expandColorVertical", 
    "options": {
        "target": "FFFFFF",
        "base": "FF0000"
    }
}

v1.01:Expands target color in image over the background color by 1 pixel horizontally and vertically.

{
    "action": "expandColor", 
    "options": {
        "target": "FFFFFF",
        "base": "FF0000"
    }
}

v1.01:Draws a solid rectangle with the specified color with the specified bounding box.

{
    "action": "expandColor", 
    "options": {
        "color": "000000",
        "bounding_box": {
            "x1": 0, 
            "y1": 34,
            "x2": 40, 
            "y2": 400
        }
    }
}

An algorithm to isolate text based on surrounding color to help with OCR. This is used in conjuction with reduceToMultiColor, where the text color and its surrounding color are reduced first. Then when segFill runs, it will replace text colors with the base color, if it is not surrounded by black.

Field	Value	Description
"base"	string	The color to replace non-segregated colors with.
"colors"	list	List of colors to isolate.

Example code:

{
    "action": "segFill", 
    "options": {
        "colors": [
            "FFFFFF", 
            "FFDF00", 
            "EFA20C"
        ], 
        "base": "FF0000"
    }
}

Example before image:

Corresponding after image:

Runs an OCR algorithm on the image. For images to OCR correctly, they need to be first reduced to as close to white text on a black background (or vice-versa), with as little noise as possible. In the segFill "before" example given above, OCR will fail because of the white colored noise above the text. Even the after image may fail on the OCR step if the red color is not mapped to black first.
If text is found, the algorithm does not require a fully accurate OCR step to still work. If the found text is close enough to the textbox text, or close enough to an indexed text, then the algorithm will still work. To help with matching, common errors can be defined so that a character and another character that the algorithm can confused it with can be mapped to the same character so that the matching phases will still work in the case of an error. The 'miss" option specifies how many character mismatches should be tolerated for a text match to be found, where the first entry is how long the string should be to tolerate 0 mismatches, the second entry how long the string should be to tolerate 1 mismatch, and so on.
The "subs" field, is for substituting OCRed characters into the text, since the text itself may change. For example, "I have 15 shillings and 6 pence." is not a reasonable text to translate for all possible number amounts. In this case, we can put in a substution rule, where we can substitute some of text found during the OCR phase into the translation directly. The characters to substitute are given in the "sub_chars" list, while the "sub_placeholder" value defines your placeholders in the translated text to use (for example: "I have {{0}} shillings and {{1}} pence." would substitute in the first found blob of substitute characters into the {{0}} spot, and the second to the {{1}} spot, assuming the "sub_placholder" is ["{{", "}}"]).

{
    "action": "indexOCR", 
        "options": {
            "common_errors": [
                "\u00fc=u", 
                "\u00f6=o", 
                "\u00e4=a", 
                "e=c", 
                "o=0", 
                "D=)", 
                "E=F", 
                "t=l", 
                "vv=W"
            ], 
            "lang": "deu", 
            "miss": [1, 6, 20, 40], 
            "mode": 6, 
            "subs": {
                "sub_chars": [
                    "0", 
                    "1", 
                    "2", 
                    "3", 
                    "4", 
                    "5", 
                    "6", 
                    "7", 
                    "8", 
                    "9"
                ], 
                "sub_placeholder": [ "{{", "}}" ]
            }, 
            "text_colors": [
                "FFFFFF", 
                "FFDF00", 
                "EFA20C"
            ]
        }
    }
}

Takes the text found during OCR and does a closeness match between it and the text found from the expanded shortlist. If the texts are close enough, then the meta data for this enty will be marked as a match, and have the original image will use the translated textbox corresponding to the entry.

{
    "action": "diffImage", 
    "options": {
        "color": "$block.index_image"
    }
}

{
    "force_index": [0,2]
}

{
  "box_attributes": [
    {
      "font size": null,
      "font color": "D3B251"
    },
    {
      "font size": 7,
      "font color": "CFE7DB"
    }
 
  ]
}

{
  "box_meta": [
    {
      "meta_req": [{"equal_variable": {"intro1":""}}, {"equal_variable": {"intro_box1":""}}],
      "meta_commands": [
        {
         "set_variable": {"unset_intro": "", "intro_box1": "true"}
        },
        {
         "set_variable": {"intro1": "true"}
        },
        {
         "disable_index": "deu_ocr"
        },
        {
          "textbox": {"req": [{"equal_variable": {"unset_intro": ""}}],
                      "delay": 7,
                      "duration": 4.5,
                      "bounding_box": [183, 345, 280, 32],
                      "font_size": 8,
                      "text": {"en": "Eric \"Speedy\" MacDoughan is a space pilot."}
          }
        },
        {
          "textbox": {"req": [{"equal_variable": {"unset_intro": ""}}],
                      "delay": 12,
                      "duration": 11,
                      "bounding_box": [183, 313, 280, 80],
                      "font_size": 8,
                      "text": {"en": "With a small career and freelance freight jobs, he earns his meager money somewhere far out in the fringes of the Milky Way."}
          }
        }
      ]
    }
  ]  
}