8000 Compare each word in lexicon to LTS results · Issue #97 · dhowe/RiTaV1 · GitHub
[go: up one dir, main page]

Skip to content
This repository was archived by the owner on Dec 9, 2025. It is now read-only.
This repository was archived by the owner on Dec 9, 2025. It is now read-only.

Compare each word in lexicon to LTS results #97

Open

Description

@dhowe
  1. Loop through the lexicon1000, print the phonemes from lexicon, vertically aligned with phonemes from LTS
about: ah b-aw1-t
       ah b-ow1-t
act: ae1-k-t
...
  1. Post a link to the results below

Activity

  1. self-assigned this
    on Apr 19, 2015
  2. added this to the NextTasks milestone on Apr 19, 2015
  3. modified the milestones: NextTasks, on Nov 16, 2016
  4. assigned and unassigned on Nov 16, 2016
  5. cqx931 commented on Apr 21, 2017

    @cqx931
    Collaborator

    http://chenqianxun.com/resources/dictLtsDifference.txt
    Total: 305 words out of 964 words have results from lexicon and LTS
    Issues:

    1. LTS will add stress 0 to unstressed syllables if a word consists of more than one syllable, for this result, I already removed all the 0 from the LTS result. It would be 475/964 words in different if I don't remove the zero.
    2. There are a few cases when there is no vowel in LTS phonemes: (mostly related to single 'e')
    noun       :n-aw1-n
                n-n
    red        :r-eh1-d
                r-d
    touch      :t-ah1-ch
                t-ch
    led        :l-eh1-d
                l-d
    through    :th-r-uw1
                th-r
    yes        :y-eh1-s
                y-z
    great      :g-r-ey1-t
                g-r-t
    say        :s-ey1
                s
    sure       :sh-uh1-r
                sh-r
    be         :b-iy1
                b
    bed        :b-eh1-d
                b-d
    she        :sh-iy1
                sh
    he         :hh-iy1
                hh
    break      :b-r-ey1-k
                b-r-k
    me         :m-iy1
                m
    we         :w-iy1
                w
    
    a          :ey1
                ey
    above      :ah b-ah1-v
                ah b-ow1-v
    again      :ah g-eh1-n
                ae1 g-eh1-n
    against    :ah g-eh1-n-s-t
                ae1 g-eh1-n-s-t
    ago        :ah g-ow1
                aa1 g-ow
    allow      :ah l-aw1
                ae1 l-ow
    also       :ao1-l s-ow
                ae1 l-s-ow
    always     :ao1-l w-ey-z
                ae-l w-ey1-z
    am         :ae1-m
                ah-m
    among      :ah m-ah1-ng
                ah m-ao1-ng
    an         :ae1-n
                ah-n
    anger      :ae1-ng g-er
                ae1 ng-er
    are        :aa1-r
                aa-r
    area       :eh1 r-iy ah
                aa r-iy1-ah
    arrange    :er ey1-n-jh
                er-ey1-n-jh
    arrive     :er ay1-v
                er-ay1-v
    as         :ae1-z
                ah-z
    atom       :ae1 t-ah-m
                ae1 t-aa1-m
    be         :b-iy1
                b
    beauty     :b-y-uw1 t-iy
                b-t-iy
    bed        :b-eh1-d
                b-d
    before     :b-ih f-ao1-r
                b-ah f-ao1-r
    begin      :b-ih g-ih1-n
                b-eh1 jh-ih1-n
    behind     :b-ih hh-ay1-n-d
                b-eh1 hh-ay1-n-d
    believe    :b-ih l-iy1-v
                b-ah l-iy1-v
    between    :b-ih t-w-iy1-n
                b-eh1 t-w-iy1-n
    body       :b-aa1 d-iy
                b-ow1 d-iy
    break      :b-r-ey1-k
                b-r-k
    busy       :b-ih1 z-iy
                b-y-ah s-iy
    but        :b-ah1-t
                b-ah-t
    by         :b-ay1
                b-iy
    can        :k-ae1-n
                k-ah-n
    carry      :k-ae1 r-iy
                k-eh1 r-iy
    century    :s-eh1-n ch-er iy
                s-eh1-n t-er-iy
    certain    :s-er1 t-ah-n
                s-er1 t-ey1-n
    character  :k-eh1 r-ih-k t-er
                ch-eh1 r-ah-k t-er
    chick      :ch-ih1-k
                ch-ih-k
    chief      :ch-iy1-f
                ch-ah-f
    choose     :ch-uw1-z
                ch-uw1-s
    city       :s-ih1 t-iy
                s-ay1 t-iy
    clear      :k-l-ih1-r
                k-l-ih1-er
    climb      :k-l-ay1-m
                k-l-ih1-m
    close      :k-l-ow1-s
                k-l-ow1-z
    column     :k-aa1 l-ah-m
                k-aa1 l-ah-m-n
    company    :k-ah1-m p-ah n-iy
                k-ah-m p-ah n-iy
    compare    :k-ah-m p-eh1-r
                k-ah-m p-aa1-r
    consonant  :k-aa1-n s-ah n-ah-n-t
                k-ah-n s-ah n-ah-n-t
    continent  :k-aa1-n t-ah n-ah-n-t
                k-ah-n t-ah n-ah-n-t
    control    :k-ah-n t-r-ow1-l
                k-aa1-n t-r-aa-l
    correct    :k-er eh1-k-t
                k-ao1-er-eh1-k-t
    cotton     :k-aa1 t-ah-n
                k-aa1-ah-n
    could      :k-uh1-d
                k-ow1-l-d
    country    :k-ah1-n t-r-iy
                k-aw1-ah1-n t-r-iy
    cow        :k-aw1
                k-ow
    create     :k-r-iy ey1-t
                k-r-iy-ey1-t
    cry        :k-r-ay1
                k-r-iy
    current    :k-er1 ah-n-t
                k-er1-ah-n-t
    dad        :d-ae1-d
                d-ae-d
    danger     :d-ey1-n jh-er
                d-ey1 ng-er
    decide     :d-ih s-ay1-d
                d-ah s-ay1-d
    depend     :d-ih p-eh1-n-d
                d-ah p-eh1-n-d
    describe   :d-ih s-k-r-ay1-b
                d-ah-s k-r-ay1-b
    desert     :d-eh1 z-er-t
                d-ah z-er-t
    design     :d-ih z-ay1-n
                d-ah z-ay1-n
    dictionary :d-ih1-k sh-ah n-eh r-iy
                d-ih1-k sh-ah n-eh1 r-iy
    die        :d-ay1
                d-iy1
    difficult  :d-ih1 f-ah k-ah-l-t
                d-ih1 f-ah k-ah1-l-t
    direct     :d-er eh1-k-t
                d-er-eh1-k-t
    discuss    :d-ih s-k-ah1-s
                d-ih1-s k-ah1-s
    distant    :d-ih1 s-t-ah-n-t
                d-ih1-s t-ah-n-t
    do         :d-uw1
                d-ow
    dog        :d-ao1-g
                d-aa1-g
    dress      :d-r-eh1-s
                d-r-eh1-s-s
    dry        :d-r-ay1
                d-r-iy
    effect     :ih f-eh1-k-t
                ah f-eh1-k-t
    eight      :ey1- t
                ey1-t
    electric   :ih l-eh1-k t-r-ih-k
                ah l-eh1-k t-r-ih-k
    element    :eh1 l-ah m-ah-n-t
                eh1 l-eh1 m-ah-n-t
    engine     :eh1-n jh-ah-n
                eh1-n jh-iy1-n
    enough     :ih-n ah1-f
                ah n-aw1
    equate     :ih k-w-ey1-t
                iy1 k-w-ah-t
    especially :ah s-p-eh1-sh l-iy
                eh1-s p-eh1 sh-ah l-iy
    even       :iy1 v-ih-n
                iy1 v-ah-n
    evening    :iy1-v n-ih-ng
                iy1 v-ah n-ih-ng
    event      :ih v-eh1-n-t
                iy1 v-eh1-n-t
    every      :eh1 v-er iy
                eh1 v-er-iy
    exercise   :eh1-k s-er s-ay-z
                eh1-g z-er s-ay1-s
    expect     :ih-k s-p-eh1-k-t
                ih-k-s p-eh1-k-t
    experience :ih-k s-p-ih1 r-iy ah-n-s
                ih-k-s p-ih1 r-iy-ah-n-s
    experiment :ih-k s-p-eh1 r-ah m-ah-n-t
                ih-k-s p-eh1 r-ah m-ah-n-t
    family     :f-ae1 m-ah l-iy
                f-ah m-ah l-iy
    famous     :f-ey1 m-ah-s
                f-ah m-ah-s
    few        :f-y-uw1
                f-uw
    finger     :f-ih1-ng g-er
                f-ih1 ng-er
    fire       :f-ay1 er
                f-ay1-er
    floor      :f-l-ao1-r
                f-l-uh1-r
    flower     :f-l-aw1 er
                f-l-aw1-er
    four       :f-ao1-r
                f-aw1-r
    from       :f-r-ah1-m
                f-r-aa1-m
    fun        :f-ah1-n
                f-ah-n
    garden     :g-aa1-r d-ah-n
                g-aa1 r-d-ah-n
    gas        :g-ae1-s
                g-ah-z
    general    :jh-eh1 n-er ah-l
                jh-eh1 n-er-ah-l
    get        :g-eh1-t
                g-ah-t
    give       :g-ih1-v
                g-ay1-v
    go         :g-ow1
                g-ow
    great      :g-r-ey1-t
                g-r-t
    half       :hh-ae1-f
                hh-ao1-f
    has        :hh-ae1-z
                hh-ah-z
    have       :hh-ae1-v
                hh-ey1-v
    he         :hh-iy1
                hh
    her        :hh-er
                hh-er1
    his        :hh-ih1-z
                hh-ih1-s
    history    :hh-ih1 s-t-er iy
                hh-ih1-s t-ao1 r-iy
    hour       :aw1 er
                hh-aw1-r
    how        :hh-aw1
                hh-ow1
    hurry      :hh-er1 iy
                hh-er1-iy
    ice        :ay1-s
                ah-s
    idea       :ay d-iy1 ah
                ay d-iy-ah
    imagine    :ih m-ae1 jh-ah-n
                ih1 m-ae1 jh-iy1-n
    in         :ih-n
                ih1-n
    include    :ih-n k-l-uw1-d
                ih1-n k-l-uw1-d
    indicate   :ih1-n d-ah k-ey-t
                ih1-n d-ah k-ey1-t
    industry   :ih1-n d-ah-s t-r-iy
                ih1-n d-ah1-s t-r-iy
    insect     :ih1-n s-eh-k-t
                ih1-n s-eh1-k-t
    instant    :ih1-n s-t-ah-n-t
                ih1-n-s t-ah-n-t
    instrument :ih1-n s-t-r-ah m-ah-n-t
                ih1-n-s t-r-ah m-ah-n-t
    invent     :ih-n v-eh1-n-t
                ih1-n v-eh1-n-t
    iron       :ay1 er-n
                ay1-er-aa1-n
    island     :ay1 l-ah-n-d
                ih1 l-ae1-n-d
    key        :k-iy1
                k-iy
    know       :n-ow1
                n-aw1
    laugh      :l-ae1-f
                l-ae1
    led        :l-eh1-d
                l-d
    leg        :l-eh1-g
                l-ah-g
    less       :l-eh1-s
                l-eh1-s-s
    let        :l-eh1-t
                l-ah-t
    lie        :l-ay1
                l-iy1
    liquid     :l-ih1 k-w-ah-d
                l-ih1 k-w-ih1-d
    listen     :l-ih1 s-ah-n
                l-ih1-s t-ah-n
    log        :l-ao1-g
                l-aa1-g
    lost       :l-ao1-s-t
                l-aa1-s-t
    low        :l-ow1
                l-ow
    machine    :m-ah sh-iy1-n
                m-ih sh-iy1-n
    major      :m-ey1 jh-er
                m-ah jh-er
    man        :m-ae1-n
                m-ah-n
    many       :m-eh1 n-iy
                m-ae1 n-iy
    market     :m-aa1-r k-ah-t
                m-aa1 r-k-ah-t
    master     :m-ae1 s-t-er
                m-ae1-s t-er
    material   :m-ah t-ih1 r-iy ah-l
                m-ah t-ih1 r-iy-ah-l
    may        :m-ey1
                m-ey
    me         :m-iy1
                m
    method     :m-eh1 th-ah-d
                m-eh1 th-aa1-d
    million    :m-ih1 l-y-ah-n
                m-ih1-l y-ah-n
    molecule   :m-aa1 l-ah k-y-uw-l
                m-ah l-ah k-y-uw1-l
    month      :m-ah1-n-th
                m-aa1-n-th
    most       :m-ow1-s-t
                m-aa1-s-t
    much       :m-ah1-ch
                m-ah-ch
    multiply   :m-ah1-l t-ah p-l-ay
                m-ah1 l-t-ah p-l-iy
    my         :m-ay1
                m-iy
    nation     :n-ey1 sh-ah-n
                n-ae1 sh-ah-n
    natural    :n-ae1 ch-er ah-l
                n-ae1 ch-er-ah-l
    nature     :n-ey1 ch-er
                n-ae1 ch-er
    necessary  :n-eh1 s-ah s-eh r-iy
                n-ah s-ah s-eh1 r-iy
    new        :n-uw1
                n-uw
    nine       :n-ih1-n
                n-ay1-n
    no         :n-ow1
                n-ow
    noun       :n-aw1-n
                n-n
    now        :n-aw1
                n-ow
    numeral    :n-uw1 m-er ah-l
                n-uw1 m-er-ah-l
    object     :aa1-b jh-eh-k-t
                ah-b jh-eh1-k-t
    occur      :ah k-er1
                aa1 k-er1
    of         :ah1-v
                ao-f
    off        :ao1-f
                ao-f
    offer      :ao1 f-er
                ah f-er
    office     :ao1 f-ah-s
                ah f-ah-s
    often      :ao1 f-ah-n
                aa1 f-ah-n
    oh         :ow1
                ow-hh
    on         :aa1-n
                ah-n
    once       :w-ah1-n-s
                aa1-n-s
    one        :w-ah1-n
                ow1-n
    only       :ow1-n l-iy
                aa1-n l-iy
    operate    :aa1 p-er ey-t
                aa1 p-er-ey1-t
    opposite   :aa1 p-ah z-ah-t
                aa1 p-ah z-ay1-t
    order      :ao1-r d-er
                ao1 r-d-er
    organ      :ao1-r g-ah-n
                ao1 r-g-ah-n
    original   :er ih1 jh-ah n-ah-l
                er-ih1 jh-ah n-ah-l
    our        :aw1 er
                aw1-er1
    own        :ow1-n
                aw1-n
    oxygen     :aa1-k s-ah jh-ah-n
                aa1-k s-ah g-ah-n
    paragraph  :p-ae1 r-ah g-r-ae-f
                p-eh1 r-ah g-r-ae1-f
    parent     :p-eh1 r-ah-n-t
                p-r-ah-n-t
    particular :p-er t-ih1 k-y-ah l-er
                p-aa1 r-t-ih1 k-y-ah l-er
    party      :p-aa1-r t-iy
                p-aa1 r-t-iy
    pay        :p-ey1
                p-ey
    perhaps    :p-er hh-ae1-p-s
                p-er1 hh-ae1-p-s
    period     :p-ih1 r-iy ah-d
                p-ih1 r-iy-ah-d
    planet     :p-l-ae1 n-ah-t
                p-l-ae1 n-eh1-t
    please     :p-l-iy1-z
                p-l-iy1-s
    plural     :p-l-uh1 r-ah-l
                p-l-er-ah-l
    poem       :p-ow1 ah-m
                p-ow1-eh1-m
    populate   :p-aa1 p-y-ah l-ey-t
                p-aa1 p-y-ah l-ey1-t
    position   :p-ah z-ih1 sh-ah-n
                p-aa1 z-ih1 sh-ah-n
    power      :p-aw1 er
                p-aw1-er
    prepare    :p-r-iy p-eh1-r
                p-r-iy p-aa1-r
    present    :p-r-eh1 z-ah-n-t
                p-r-eh1 z-eh1-n-t
    pretty     :p-r-ih1 t-iy
                p-r-eh1 t-iy
    process    :p-r-aa1 s-eh-s
                p-r-aa1 s-eh1-s
    product    :p-r-aa1 d-ah-k-t
                p-r-ah d-ah1-k-t
    put        :p-uh1-t
                p-ah1-t
    quick      :k-w-ih1-k
                k-w-ih-k
    quiet      :k-w-ay1 ah-t
                k-w-ay1-ah-t
    quotient   :k-w-ow1 t-ih ah-n-t
                k-w-ow1 t-ah-eh1-n-t
    radio      :r-ey1 d-iy ow
                r-ey1 d-iy-ow
    range      :r-ey1-n-jh
                r-ae1-n-jh
    read       :r-eh1-d
                r-iy1-d
    ready      :r-eh1 d-iy
                r-iy1 d-iy
    reason     :r-iy1 z-ah-n
                r-iy1 s-ah-n
    red        :r-eh1-d
                r-d
    remember   :r-ih m-eh1-m b-er
                r-ah m-eh1-m b-er
    repeat     :r-ih p-iy1-t
                r-ah p-iy1-t
    reply      :r-ih p-l-ay1
                r-iy p-l-iy
    represent  :r-eh p-r-ah z-eh1-n-t
                r-eh1 p-r-ah z-eh1-n-t
    require    :r-iy k-w-ay1 er
                r-eh1 k-w-ay1-r
    result     :r-ih z-ah1-l-t
                r-ah z-ah1-l-t
    row        :r-ow1
                r-ow
    said       :s-eh1-d
                s-ey1-d
    say        :s-ey1
                s
    school     :s-k-uw1-l
                sh-k-uw1-l
    science    :s-ay1 ah-n-s
                s-ay1-ah-n-s
    sea        :s-iy1
                s-iy-ah
    second     :s-eh1 k-ah-n-d
                s-eh1 k-aa1-n-d
    separate   :s-eh1 p-er ey-t
                s-eh1 p-er-ah-t
    several    :s-eh1 v-r-ah-l
                s-eh1 v-er-ah-l
    she        :sh-iy1
                sh
    shoe       :sh-uw1
                sh-ow1
    should     :sh-uh1-d
                sh-ow1-l-d
    shoulder   :sh-ow1-l d-er
                sh-ow1 l-d-er
    show       :sh-ow1
                sh-aw1
    sign       :s-ay1-n
                s-iy1-n
    silver     :s-ih1-l v-er
                s-ih1 l-v-er
    sister     :s-ih1 s-t-er
                s-ih1-s t-er
    sky        :s-k-ay1
                s-k-iy
    snow       :s-n-ow1
                s-n-aw1
    so         :s-ow1
                s-ow
    soft       :s-aa1-f-t
                s-ao1-f-t
    soldier    :s-ow1-l jh-er
                s-ow1 l-d-iy-er
    some       :s-ah1-m
                s-ah-m
    son        :s-ah1-n
                s-ah-n
    square     :s-k-w-eh1-r
                s-k-w-aa1-r
    study      :s-t-ah1 d-iy
                s-t-uw1 d-iy
    substance  :s-ah1-b s-t-ah-n-s
                s-ah1-b-s t-ah-n-s
    suffix     :s-ah1 f-ih-k-s
                s-ah f-ih-k-s
    sugar      :sh-uh1 g-er
                s-uh1 g-er
    suggest    :s-ah-g jh-eh1-s-t
                s-ah-g jh-ah-s-t
    sun        :s-ah1-n
                s-ah-n
    supply     :s-ah p-l-ay1
                s-ah p-l-iy
    sure       :sh-uh1-r
                sh-r
    surface    :s-er1 f-ah-s
                s-er1 f-ey1-s
    system     :s-ih1 s-t-ah-m
                s-ih1-s t-ah-m
    ten        :t-eh1-n
                t-ah-n
    than       :dh-ae1-n
                th-ae1-n
    that       :dh-ae1-t
                th-ae1-t
    their      :dh-eh1-r
                th-eh1-ih1-r
    them       :dh-eh1-m
                th-eh1-m
    then       :dh-eh1-n
                th-ah-n
    there      :dh-eh1-r
                th-ih1-r
    these      :dh-iy1-z
                th-iy1-z
    they       :dh-ey1
                dh-iy
    thick      :th-ih1-k
                th-ih-k
    thing      :th-ih1-ng
                th-ih-ng
    those      :dh-ow1-z
                th-ow1-z
    though     :dh-ow1
                th-ao1
    thousand   :th-aw1 z-ah-n-d
                th-aw1 s-ah-n-d
    through    :th-r-uw1
                th-r
    throw      :th-r-ow1
                th-r-ow
    thus       :dh-ah1-s
                dh-ah-s
    tie        :t-ay1
                t-iy1
    tire       :t-ay1 er
                t-ay1-er
    to         :t-uw1
                t-ow
    touch      :t-ah1-ch
                t-ch
    toward     :t-ah w-ao1-r-d
                t-ow1-er-d
    triangle   :t-r-ay1 ae-ng g-ah-l
                t-r-iy-ae1-ng g-ah-l
    try        :t-r-ay1
                t-r-iy
    two        :t-uw1
                t-w-ow
    unit       :y-uw1 n-ah-t
                y-uw1 n-ih1-t
    us         :ah1-s
                ah-s
    use        :y-uw1-s
                y-uw1-z
    value      :v-ae1-l y-uw
                v-ae1 l-uw1
    very       :v-eh1 r-iy
                v-r-iy
    village    :v-ih1 l-ah-jh
                v-iy l-ih-jh
    vowel      :v-aw1 ah-l
                v-aa1-ah-l
    we         :w-iy1
                w
    were       :w-er
                w-ih1-r
    what       :w-ah1-t
                w-ae1-t
    when       :w-eh1-n
                w-ah-n
    which      :w-ih1-ch
                w-ih-ch
    who        :hh-uw1
                hh-ow
    whose      :hh-uw1-z
                hh-ow1-z
    why        :w-ay1
                w-iy
    wire       :w-ay1 er
                w-ay1-r
    with       :w-ih1-dh
                w-ih1-th
    would      :w-uh1-d
                w-ow1-l-d
    yes        :y-eh1-s
                y-z
    your       :y-ao1-r
                y-aw1-r
    
  6. dhowe commented on Apr 21, 2017

    @dhowe
    OwnerAuthor

    Can we generalize any rules here? Differences that occur every time?

    One thing we should programmatically fix for sure if we can is the LTS problem with missing vowels in single-syllable words

    Also, we need to remove 0s from all LTS results (in the code)

  7. cqx931 commented on Apr 21, 2017

    @cqx931
    Collaborator

    The dictionary is more acurate, I don't think we should replace the entry with the result from LTS.
    The chance of getting different result is roughly 1/3 for the whole dictionary(28,000);

    Easy to fix:
    1.Remove 0 from LTS result
    2.if there is only one syllable, add stress to the vowel phoneme

    What LTS get's wrong:
    vowel
    stress
    syllables
    minor consonant case

    Some statics of the wrong pronunciation in phoneme: (dict -> LTS,cases > 200, in whole dictionary)

    ih -> ah:2561
    ah -> aa:363
    aa -> ah:342
    ah -> ae:326
    ih -> eh:305
    eh -> ah:293
    ah -> ih:292
    ae -> ah:283
    iy -> ah:246
    ah -> eh:209
    

    LTS got many phonemes to ah

    ih -> ah:2561 
    aa -> ah:342
    eh -> ah:293
    ae -> ah:283
    iy -> ah:246
    ey -> ah:187
    ay -> ah:108
    

    Example Analysis for the case ih -> ah
    Case1: ends with ed[ex:abated t-ih-d -> t-ah-d];
    Case2: ends with es [acknowledges];
    Case3:starts with be [betray];
    Case4:starts with re [resemble];
    Case5:starts with pre [presume];
    Case6:starts with se [seduce];
    Case7:contains nist [pianist,administration];
    Case8:ends with ness [friendliness]
    Case9:ends with ility [imposibility]
    ...

    First try to do some analysis of the result, if this direction seems fine to you, I will keep update more later.
    Also, I'm quite surprised to see so many basic prefix or suffix in the cases above...

  8. dhowe commented on Apr 21, 2017

    @dhowe
    OwnerAuthor

    Easy to fix:
    1.Remove 0 from LTS result
    2.if there is only one syllable, add stress to the vowel phoneme

    Lets do these to start, and push the rest to 1.7

  9. modified the milestones: , on Apr 24, 2017
  10. cqx931 commented on Apr 24, 2017

    @cqx931
    Collaborator

    1.6 part is finished, modified the milestone to Version 1.7

  11. cqx931 commented on May 16, 2017

    @cqx931
    Collaborator

    One thing we should programmatically fix for sure if we can is the LTS problem with missing vowels in single-syllable words

    I think we might want to be more specific with the situation we want to improve with the lts result.
    Do we want to improve the lts result for words that are contained in the full lexicon but not in lexicon1000? Or do we want to improve the lts result for words in lexicon 1000?

    When LTS rules are trained, they are not aimed for pronouncing single-syllable words. Words less than four letters are suggested to be removed from the training set. So it makes sense why LTS rules get so many single-syllable words wrong

    For LTS rules, the 'ed' in "yelled" and "red" are treated in the same way, that's why the vowel is missing. Same for "yes" and "she".
    One way is to create special word set that includes pronunciation of these short common words, because they are less regular.

  12. dhowe commented on May 16, 2017

    @dhowe
    OwnerAuthor

    you mean add them to the lexicon?

  13. cqx931 commented on May 17, 2017

    @cqx931
    Collaborator

    These words are already in the full lexicon and the 1k lexicon. So the only case when LTS would be used for them is when LTS works with no Lexicon attached. If we do want to get them right for this scenario, then the pronounciation would be stored as part of the LTS code.
    However, we don't have this case according to the size specification we have currently:

    rita.js (core-only, no lexicon) -> tiny
    rita-small.js (1k-word lexicon + lts) -> medium
    rita-full.js (35k-word-lexicon + lts) -> large
    

    You talked about merging RiLexicon to main rita.js, so I'm not sure whether we will face this case for 'tiny' in the future(core, RiLexicon,LTS, no lexicon)

  14. dhowe commented on May 17, 2017

    @dhowe
    OwnerAuthor

    If the LTS and part-of-speech are correct/non-ambiguous, then such words could be removed from the lexicon (though I suppose there could be a performance hit for doing the LTS vs. the fast lexicon lookup)

  15. cqx931 commented on May 17, 2017

    @cqx931
    Collaborator

    So the aim of the comparison is to remove pronunciation entries from the lexicon? For both Full lexicon and the 1k lexicon?

  16. dhowe commented on May 18, 2017

    @dhowe
    OwnerAuthor

    Two aims: 1) to get the outputs matching; 2) to remove unnecessary entries from lexicons (entries whose pronunciation data matches the LTS engine, and whose POS can be correctly guessed, primarily 'nn')

  17. cqx931 commented on May 18, 2017

    @cqx931
    Collaborator

    Just did a quick check in the current full lexicon,

    1. 10171 out of 28095 entries (36.2%) got the same pronunciation result from LTS and from the lexicon
    2. There are 2811 words whose POS tag only contains 'nn' and the pronunciation data from LTS engine matches with what we have in the lexicon. These should be the safe words to delete in terms of Pronunciation and POS tags.
  18. modified the milestones: Version 1.3, Version 1.4 on Jun 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    0