Re: Corpora: closed class word list

From: Douglas Rohde (dr@tedlab.mit.edu)
Date: Tue Jun 04 2002 - 18:12:26 MET DST

  • Next message: Gil Graf: "Corpora: control chars"

    Diego Molla wrote:

    > By definition, a list of closed class words must be easy to compile,
    > since new additions to the list would be rare.
    >
    > Oddly enough, I haven't found any such list on the Web. A student of
    > mine needs to use a list of closed class words. Does anybody know of
    > such a list?

    Assuming you're interested in English, I have a list of closed class
    words that I developed for working with a corpus of usenet text. It has
    about 150 words. As far as I can tell, the set of closed class words in
    English is not completely well-defined. Some words (pronouns,
    conjunctives, articles) are clearly closed class. But certain adverbs
    and common verbs are probably debatable, as are, I think, digits. So
    for what it's worth, here's my list. You notice that it includes things
    like punctuation and stuff in brackets like <NUM> (which stands for a
    number) that you may want to remove.

    Doug


    .
    ,
    THE
    TO
    AND
    A
    OF
    <MIX>
    "
    IN
    I
    <NUM>
    :
    YOU
    IS
    THAT
    )
    (
    IT
    FOR
    ON
    !
    <URL>
    HAVE
    WITH
    ?
    THIS
    BE
    ...
    NOT
    ARE
    AS
    WAS
    BUT
    OR
    FROM
    MY
    AT
    IF
    THEY
    <XXX>
    YOUR
    ALL
    HE
    BY
    ONE
    ME
    WHAT
    SO
    CAN
    WILL
    DO
    AN
    ABOUT
    WE
    JUST
    WOULD
    THERE
    NO
    LIKE
    OUT
    HIS
    HAS
    UP
    MORE
    WHO
    WHEN
    DON'T
    SOME
    HAD
    THEM
    ANY
    THEIR
    IT'S
    ONLY
    ;
    WHICH
    I'M
    BEEN
    OTHER
    WERE
    HOW
    THEN
    NOW
    HER
    THAN
    SHE
    WELL
    <IPA>
    ALSO
    US
    VERY
    BECAUSE
    AM
    HERE
    COULD
    EVEN
    <EMO>
    HIM
    INTO
    OUR
    MUCH
    TOO
    DID
    SHOULD
    OVER
    WANT
    THESE
    MAY
    WHERE
    MOST
    MANY
    THOSE
    DOES
    WHY
    PLEASE
    OFF
    GOING
    ITS
    I'VE
    DOWN
    THAT'S
    CAN'T
    YOU'RE
    DIDN'T
    ANOTHER
    AROUND
    MUST
    <EMA>
    FEW
    DOESN'T
    EVERY
    YES
    EACH
    MAYBE
    I'LL
    AWAY
    DOING
    OH
    ELSE
    ISN'T
    HE'S
    THERE'S
    HI
    WON'T
    OK
    THEY'RE
    YEAH
    MINE
    WE'RE
    WHAT'S
    SHALL
    SHE'S
    HELLO
    OKAY
    HERE'S
    -



    This archive was generated by hypermail 2b29 : Wed Jun 05 2002 - 13:04:11 MET DST