Interest in this project

[jdferreira]

I'm not sure anyone is monitoring this project anymore, but would be interested in taking some of the code and do my own LiveScribe program.

Does anyone know how the STF Parser was created? I managed to find on an old forum a thread (http://ablivio.free.fr/forum/viewtopic.php?id=36) that seems to explain the basics of this file format, but I think the python parser is way more developed than that thread.

Also, I think the parser can still do a better job: at least on windows, the images have a nicer quality. I'm willing to improve this code, but I can't find a way to understand the parser...

Furthermore, does anyone know the use of the other files from the pen? There are .area and .gfx files

[yonran]

Hi @jotomicron. I did not make the original parser and have a hard time comprehending the decode function. But I did find a simple bug in reading stroke data; perhaps that fixes your issue?

I don’t know what you would use the page.area and page.gfx xml files for.

[yonran]

Hi @jotomicron. I have had some success combining the XML files with PNG files within the AFD archives to get backgrounds for the stroke files:

1. Open main.info to read the page address of the first page: pagestart: 2208.0.0.27
2. Open main.document, which contains a list of page elements e.g. <page basepath="pages/2" areasfile_ref="page.areas" gfxfile_ref="page.gfx" width="6773" height="8127" haspattern="1" hidden="0"/>.
3. Open pages/2/page.gfx to read the dimensions of the background image and the pattern area.
   • <image elemid="0x1" x="0.000" y="0.000" width="6773.000" height="8127.000" src="pages/resources/LS_SA0_01_INT_P001.eps" page="0"/>
   • Not sure how to interpret <drawingarea elemid="0x2" x="0.000" y="0.000" width="6773.000" height="8127.000"/> yet. For the A4 and A5 notebooks, x and y are negative. I don’t know whether the coordinates in the STF files will be relative to the top left of the page, or the top left of the pattern.
4. I couldn’t find any eps files, but I used userdata/lsac_data/LS_SA0_01_INT_P001.png. It has to be stretched e.g. 1500×1800 → 6773×8127 according to the image element above.
5. Now, given that pagestart: 2208.0.0.27 and you want to know the background for page 2208.0.1.19, you need to find out which <page/> it corresponds to in main.document. I believe that the rule is that each <page haspattern="1"/> gets a new page address. So the first element containing haspattern="1" has page address=2208.0.0.27, the second one gets page address=2208.0.0.28, etc., and the 101st page with haspattern="1" gets page address=2208.0.1.19 (=string_from_page_address(paAdd(page_address_from_string("2208.0.0.27")))). You have to line them up properly, or else you’ll get the wrong background image for a given page.

The functions page_address_from_string, string_from_page_address, paAdd may be of use:

def page_address_from_string(s):
    pa = 0
    parts = s.split('.')
    if not (4 <= len(parts) <= 5):
        raise Exception("Page address must have 4 or 5 parts, not %d: %s", len(parts), s)
    long_digit_index = len(parts) - 3  # third least significant part
    for i,part in enumerate(parts):
        pa *= 0x10000 if i == long_digit_index else 0x1000
        expected_max = 0xffff if i == long_digit_index else 0xfff
        x = int(part, 10)
        if not (0 <= x <= expected_max):
           raise Exception("digit %d of page address %s exceeds range [0,%d]" % ( i, s, expected_max))
        pa += x
    return pa
def string_from_page_address(x):
    parts = []
    if not (0 <= x <= 1<<64):
        raise Exception("PA must be u64; got %x", x)
    i = 0
    while x != 0 or i < 4:
        num_bits = 16 if i == 2 else 12
        parts.append(str(x & ((1 << num_bits) - 1)))
        x = x >> num_bits
        i += 1
    return '.'.join(reversed(parts))
tab1_indexes = [
    1,0x16,0x13,0x18,0x15,0x12,0x20,0x19,
    0x16,0x20,0x20,0x20,0x20,0x0E,5,0x1C,
    0x0F,0x0A,5,0x1C,7,2,2,0x1F,
    0,0x0B,0x12,6,6,0x1B,0x11,0x0D,
    8,0x14,0x0C,2,0x10,0x1D,0x0C,0x1E,
    9,0x12,3,0x14,0x17,0x12,0x12,0x12,
    2,0x20,2,5,2,0x1C,2,0x20,
    0x20,0x20,0x20,0x20,0x1A,5,0x1C,0x20,
    0x20,0x20,0x20,0x20,0x20,0x20,0x1C,0x1C,
    2,2,5,5,0x20,0x20,0x20,0x20,
    0x20,2,2,2,2,2,2,2,
    2,2,2,5,5,5,5,5,
    4,0x20,0x20,0x20,0x20,0x20,0x20,0x20,
    0x20,0x20,0x20,0x20,0x20,0x20,0x1A,5,
    0x1C,0x20,0x20,0x20,0x20,0x20,0x20,0x20,
    0x20,0x20,0x20,0x20,0x20,0x20,0x20,0x20,
    0x20,0x20,0x20,0x20,0x20,0x20,0x20,0x20,
    0x20,0x20,0x20,0x20,0x20,0x20,0x20,0x20,
    2,2,2,2,2,2,2,2,
    2,2,0x1C,0x1C,0x1C,0x1C,0x1C,0x1C,
    0x1C,0x1C,0x1F,0x1F,0x1F,0x1F,0x1F,0x1F,
    2,2,2,2,2,0x20,0x20,0x20,
    0x20,0x20,0x20,0x20,0x20,0x20,0x20,0x20,
    0x20,0x20,0x20,0x20,0x20,0x20,0x20,0x20]
tab1 = [
    (0x0028,0x0a8c,0x0a8c),
    (0x0040,0x2000,0x2000),
    (0x0100,0x0532,0x0955),
    (0x0100,0x0444,0x0811),
    (0x09d5,0x0682,0x0655),
    (0x0020,0x0874,0x0b24),
    (0x0080,0x03be,0x0811),
    (0x0400,0x022e,0x0165),
    (0x0020,0x02c4,0x0394),
    (0x0020,0x0540,0x095f),
    (0x0100,0x054a,0x04b2),
    (0x0100,0x04bd,0x042e),
    (0x0020,0x0336,0x042e),
    (0x0100,0x02c4,0x0394),
    (0x0020,0x0194,0x03ec),
    (0x0080,0x038c,0x0630),
    (0x0010,0x01af,0x022b),
    (0x0080,0x0767,0x061e),
    (0x0200,0x2000,0x2000),
    (0x0020,0x031e,0x0416),
    (0x0100,0x0540,0x092a),
    (0x0400,0x0200,0x0200),
    (0x0c2a,0x0a85,0x08b2),
    (0x0100,0x0540,0x0929),
    (0x0020,0x04b2,0x054a),
    (0x0080,0x042c,0x0c05),
    (0x0100,0x0532,0x0a29),
    (0x0020,0x06ea,0x0a95),
    (0x0020,0x10a8,0x1358),
    (0x0020,0x0394,0x06f7),
    (0x0020,0x0444,0x0811),
    (0x000a,0x0bf8,0x1300),
    (0x0000,0x0000,0x0000)]
tab2 = [
    (0x001d,0x006c,0x0428),
    (0x001f,0x006c,0x01a6),
    (0x0021,0x006c,0x0308),
    (0x0025,0x006c,0x05b9),
    (0x0029,0x006c,0x07fc),
    (0x002e,0x008b,0x0fbf),
    (0x0030,0x0080,0x2000)]
def page_address_digit_divisions(bbb):
    """Helper function for paAdd."""
    if bbb < 0xc0:
        tab1_index = tab1_indexes[bbb]
        (ddd_per_eee, inverse_ccc_per_eee, inverse_max_ccc) = tab1[tab1_index]
        if ddd_per_eee > 0:
            return (ddd_per_eee, inverse_ccc_per_eee, inverse_max_ccc)
        else:
            return None
    elif bbb >= 0x120:
        if bbb >= 0x480 and bbb < 0x8d0:
            bbbmod48 = bbb % 0x30
            for i in range(7):
                (s, ddd_per_eee, inverse_ccc_per_eee) = tab2[i]
                if bbbmod48 < s:
                    return (ddd_per_eee, inverse_ccc_per_eee, inverse_ccc_per_eee)
            return (0, 0, 0)
        else:
            return None
    elif ( (bbb < 0xc0 or bbb > 0xd7) and
        (bbb < 0xf0 or bbb > 0x107) ):
        if ( (bbb < 0xd8 or bbb > 0xe3) and
            (bbb < 0x108 or bbb > 0x113)):
            if ( (bbb < 0xe4 or bbb > 0xef) and (bbb < 0x114 or bbb > 0x11f) ):
                return (0, 0, 0)
            else:
                return (0x20, 0x874, 0xb24)
        else:
            return (0x20, 0x10a8, 0x1358)
    else:
        return (0x100, 0x532, 0x955)
def paAdd(page_address, amount):
    aaa = (page_address & 0xfff0000000000000) >> 0x34
    bbb = (page_address & 0x000fff0000000000) >> 0x28
    ccc = (page_address & 0x000000ffff000000) >> 0x18
    ddd = (page_address & 0x0000000000fff000) >> 0xc
    eee = (page_address & 0x0000000000000fff)
    if aaa != 0:
        return 0xffefffffffffffff
    divisions = page_address_digit_divisions(bbb)
    if divisions is None:
        return 0xffefffffffffffff
    (ddd_per_eee, inverse_ccc_per_eee, inverse_max_ccc) = divisions
    maxccc = 0x800000 // inverse_max_ccc
    ddd_per_ccc = 0x800000 // (ddd_per_eee * inverse_ccc_per_eee)
    eee_per_ccc = ddd_per_eee * ddd_per_ccc
    amount_cccs = amount // eee_per_ccc
    ccc = ccc + amount_cccs
    amount = amount - (amount_cccs * eee_per_ccc)
    amount_ddds = amount // ddd_per_eee
    ddd = ddd + amount_ddds
    amount = amount - ddd_per_eee * amount_ddds
    eee = eee + amount
    if eee < 0:
        eee = eee + ddd_per_eee
        ddd = ddd - 1
    elif eee >= ddd_per_eee:
        eee = eee - ddd_per_eee
        ddd = ddd + 1
    if ddd < 0:
        ddd = ddd + ddd_per_ccc
        ccc = ccc - 1
    elif ddd >= ddd_per_ccc:
        ddd = ddd - ddd_per_ccc
        ccc = ccc + 1
    if ccc < 0 or ccc >= maxccc:
        return 0xffefffffffffffff
    return (aaa << 0x38) | (bbb << 0x28) | (ccc << 0x18) | (ddd << 0xc) | eee