Cabocha: A Japanese Syntactic Parser

This is page is adapted from musprg’s Japanese post and explained in English: https://qiita.com/musaprg/items/9a572ad5c4e28f79d2ae

Installing Cabocha (harder way)

1) Make sure you download and install the CRF first

Download source for CRF++ (version 0.55 or higher) source
- https://taku910.github.io/crfpp/#download

$tar xzvf CRF++-[version#].tar.gz
$cd CRF++-[0.51]version#]
$./configure
$make
$make install

Note: make sure you move the downloaded source file to somewhere besides Dropbox. If you don’t, you may get an error like the one below

test -z "/usr/local/lib" || ./install-sh -c -d "/usr/local/lib"
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libcrfpp.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libcrfpp.0.dylib /usr/local/lib/libcrfpp.0.dylib
libtool: install: (cd /usr/local/lib && { ln -s -f libcrfpp.0.dylib libcrfpp.dylib || { rm -f libcrfpp.dylib && ln -s libcrfpp.0.dylib libcrfpp.dylib; }; })
libtool: install: /usr/bin/install -c .libs/libcrfpp.lai /usr/local/lib/libcrfpp.la
libtool: install: /usr/bin/install -c .libs/libcrfpp.a /usr/local/lib/libcrfpp.a
libtool: install: chmod 644 /usr/local/lib/libcrfpp.a
libtool: install: ranlib /usr/local/lib/libcrfpp.a
/bin/sh: /Users/auroratsai/Dropbox: No such file or directory
make[1]: *** [install-libLTLIBRARIES] Error 127
make: *** [install-am] Error 2

2) Download and Install Cabocha

http://taku910.github.io/cabocha/

I downloaded the latest version cabocha-0.69 as a tar file.

If Mac or Unix:

tar xvfj somefilename.tar.bz2
cd cabochafilename
./configure --with-mecab-config=`which mecab-config` --with-charset=UTF8
make
sudo make install

You may get an error:

Undefined symbols for architecture x86_64:
  "_mecab_lattice_add_request_type", referenced from:
      CaboCha::MorphAnalyzer::parse(CaboCha::Tree*) const in morph.o
  "_mecab_lattice_clear", referenced from:
      CaboCha::MorphAnalyzer::clearMeCabLattice(mecab_lattice_t*) in morph.o
  "_mecab_lattice_destroy", referenced from:
      CaboCha::MorphAnalyzer::deleteMeCabLattice(mecab_lattice_t*) in morph.o
  "_mecab_lattice_get_bos_node", referenced from:
      CaboCha::MorphAnalyzer::parse(CaboCha::Tree*) const in morph.o
  "_mecab_lattice_new", referenced from:
      CaboCha::MorphAnalyzer::parse(CaboCha::Tree*) const in morph.o
  "_mecab_lattice_set_sentence2", referenced from:
      CaboCha::MorphAnalyzer::parse(CaboCha::Tree*) const in morph.o
  "_mecab_lattice_strerror", referenced from:
      CaboCha::MorphAnalyzer::parse(CaboCha::Tree*) const in morph.o
  "_mecab_parse_lattice", referenced from:
      CaboCha::MorphAnalyzer::parse(CaboCha::Tree*) const in morph.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libcabocha.la] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

I wasn’t sure how to fix this, so I installed cabocha using Homebrew

The cabocha python package is located within the python folder in the Cabocha. Navigate to the folder. Then try:

Python3 setup.py build

If you get an error like the one below:

  >>> import CaboCha
 Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 ImportError: No module named 'CaboCha'

and/or

 [/Users/user/cabocha
  Traceback (most recent call last):
0.67/python]python setup.py install
  File "setup.py", line 18, in <module>
 include_dirs=cmd2("cabocha-config --inc-dir"),
File "setup.py", line 10, in cmd2
 return string.split (cmd1(str))
 AttributeError: module 'string' has no attribute 'split'

and

Traceback (most recent call last):
File "python_test2.cgi", line 5, in
import CaboCha
File "/home/username/local/lib/python2.7/site-packages/CaboCha.py", line 26, in CaboCha = swig_import_helper()
File "/home/username/local/lib/python2.7/site-packages/CaboCha.py", line 22, in swig_import_helper
mod = imp.load_module('_CaboCha', fp, pathname, description)
ImportError: Shared object "libcabocha.so.4" not found, required by "_CaboCha.so"

This is because the setup file is still set up to work with python 2. We can fix this by changing the setup.py file.

#Find the following code within the setup file:
def cmd2(str):
   return string.split (cmd1(str))
#Change to the code below:
def cmd2(str):
   return cmd1(str).split()

This fixed the problem. (See http://qiita.com/kansiho/items/8f9303d5ea8bb8046131)

Then do:

sudo python3 setup.py install

I also got this problem when trying to import cabocha into python3:

aurora@Lynne:~/python/cabocha-0.68/python$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information. >>> import CaboCha
Traceback (most recent call last):
File "/mnt/c/Users/Aurora/Dropbox/Python/cabocha-0.68/python/CaboCha.py", line 18, in swig_import_helper
fp, pathname, description = imp.find_module('_CaboCha', [dirname(__file__)]) File "/usr/lib/python3.5/imp.py", line 296, in find_module
raise ImportError(_ERR_MSG.format(name), name=name) ImportError: No module named '_CaboCha'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/c/Users/Aurora/Dropbox/Python/cabocha-0.68/python/CaboCha.py", line 28, in
<module>
_CaboCha = swig_import_helper()
File "/mnt/c/Users/Aurora/Dropbox/Python/cabocha-0.68/python/CaboCha.py", line 20, in swig_import_helper
import _CaboCha
ImportError: libcabocha.so.4: cannot open shared object file: No such file or directory

Fix: Make a symlink to libcabocha.so.4

aurora@Lynne:/usr/lib$ ls /usr/local/lib/
cabocha libcabocha.so libcrfpp.a
libcabocha.a libcabocha.so.4 libcrfpp.la libcrfpp.so.0.0.0
libcabocha.la libcabocha.so.4.0.0 libcrfpp.so python2.7 aurora@Lynne:/usr/lib$ ln -s /usr/local/lib/libcabocha.so.4 libcabocha.so.4

Install using Homebrew (Easy Method)

Install CRF++ using Homebrew:

brew install crf++

Then install cabocha

brew install cabocha

Cabocha Python wrapper

Clone the CaboCha repository and then unzip it.
CaboChaのリポジトリをクローンする必要がありますが、リポジトリサイズが100M超なのでzipで落とします。

curl -OL https://github.com/taku910/cabocha/archive/master.zip
unzip master.zip
cd cabocha-master
pip3 install python/

THe official version of Cabocha can be installed using the above directions. However, it’s quite cumbersome to use. To simplify this, kenkov made an Python3 library available here. If you’d like to use it, you can do the following instead:

公式が提供しているPythonバインディングは上記手順でインストールすることができます。ただ、公式のものは少し使い方が煩雑です。そこで、kenkovさんがPython3用のラッパーライブラリを開発しています。こちらがわりと便利なので、お好みで導入してください。 https://github.com/kenkov/cabocha 以下コマンドでインストールすることができます。

git clone https://github.com/kenkov/cabocha
pip install cabocha/

According to the Readme page on kenkov’s cabocha, you can work with cabocha using the following:
GitHubのREADMEによると、使い方は、以下の様な感じです。

import cabocha
from cabocha.analyzer import CaboChaAnalyzer
analyzer = CaboChaAnalyzer()
tree = analyzer.parse("日本語の形態素解析はすごいです。")
for chunk in tree:
  for token in chunk:
    print(token)

## Token("日本語")
## Token("の")
## Token("形態素")
## Token("解析")
## Token("は")
## Token("すごい")
## Token("です")
## Token("。")

chunks = tree.chunks
start_chunk = chunks[0]
print(start_chunk)

## Chunk("日本語の")

print(start_chunk.next_link)

## Chunk("形態素解析は")

print(start_chunk.next_link.next_link)

## Chunk("すごいです。")

Cabocha: A Japanese Syntactic Parser

Installing Cabocha (harder way)

Install using Homebrew (Easy Method)

Cabocha Python wrapper

Other Links/Sources