The scripts I use to train models are in the bin/
folder. In the past, I've also used Maven launchers.
-
Generate your classpath using
mvn dependency:build-classpath
and copy/paste the output intobin/train-conllchain-ner.sh
in the appropriate place to set theCP
variable. -
Clone FACTORIE,
cd
into the directory, and generate a jar usingmvn -Dmaven.test.skip=true package -Pnlp-jar-with-dependencies
Then copy/paste the path to the jar into
bin/train-conllchain-ner.sh
to set thefactoriejar
variable. -
Change other options as desired (see e.g.
cc.factorie.app.nlp.ner.ChainNerOpts
for available options) -
Run
$ ./bin/train-conllchain-ner.sh
Note #1: the lexicons use a significant amount of memory and you may have to modify your JVM heap size or something
Repeat the above, except using bin/optimize-conllchain-ner.sh
.
CoNLL 2003 (on blake):
- training set:
/iesl/data/conll/2003/eng.train
- dev set:
/iesl/data/conll/2003/eng.testa
- test set:
/iesl/data/conll/2003/eng.testb
Brown Clusters (on blake):
/iesl/canvas/ksilvers/data/brownBllipClusters
(I copied this from A. Passos' folder)
-
Clone FACTORIE
-
Package i.e.
mvn -Dmaven.test.skip=true package -Pnlp-jar-with-dependencies
-
Start FACTORIE using ConllChainNer:
$ bin/fac nlp --conll-chain-ner
-
Process data:
$ echo "Hello President Bush" | nc localhost 3228
See src/main/scala/edu/umass/cs/iesl/nertut/App.scala