読者です 読者をやめる 読者になる 読者になる

kestrelをインストールする

kestrelはメッセージキューの一種でtwitterでも利用されているらしい。
たまたまインストールする機会があったので作業メモ。

環境

OS: CentOS 5.5 (64bit)
jdk: 1.6.0
kestrel: http://github.com/robey/kestrel

git

kestrelのbuildはsbtってのを使うんだけど(後述)、
build処理の中でgitを呼び出してるらしく、gitがインストされていなとエラーになりました。

まぁkestrelのソース自体githubにあるのでこの機会に入れてしまいます。

# yum install git.x86_64

例によってCentOSのデフォのレポジトリでは入ってないかもしれない。
その時は、DAG(rpmforge)のレポジトリを追加もしくはソースからインストールする。

sbt

sbtはscalaのbuildツール。makeコマンドみたいなもん。
googlecodeからダウンロードできる。
インストールってゆうかコマンド作るって感じ。そのまま実行してもまぁいい。

$ wget http://simple-build-tool.googlecode.com/files/sbt-launch-0.7.4.jar
$ vi sbt
$
$ cat sbt
java -Xmx512M -jar `dirname $0`/sbt-launch-0.7.4.jar "$@"
$
$chmod u+x sbt
$

パスは適宜通す。
めんどくさい場合は

$ java -Xmx512M -jar sbt-launch-0.7.4.jar foo bar

みたく直接コマンドを作っても良い。

kestre

ソースコードはgithubにあるのでそこからダウンロード。
どうせなんでgitコマンドを使う。
インストールディレクトリは/usr/local配下とする。

# cd /usr/local/
# git clone git://github.com/robey/kestrel.git
# cd kestrel/
# sbt clean update package-dist
Getting Scala 2.7.7 ...
downloading http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.7.7/scala-compiler-2.7.7.jar ...
        [SUCCESSFUL ] org.scala-lang#scala-compiler;2.7.7!scala-compiler.jar (16737ms)
downloading http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.7.7/scala-library-2.7.7.jar ...
        [SUCCESSFUL ] org.scala-lang#scala-library;2.7.7!scala-library.jar (9260ms)
:: retrieving :: org.scala-tools.sbt#boot-scala
        confs: [default]
        2 artifacts copied, 0 already retrieved (9911kB/45ms)
Getting org.scala-tools.sbt sbt_2.7.7 0.7.4 ...
downloading http://databinder.net/repo/org.scala-tools.sbt/sbt_2.7.7/0.7.4/jars/sbt_2.7.7.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#sbt_2.7.7;0.7.4!sbt_2.7.7.jar (2382ms)
downloading http://repo1.maven.org/maven2/com/jcraft/jsch/0.1.31/jsch-0.1.31.jar ...
        [SUCCESSFUL ] com.jcraft#jsch;0.1.31!jsch.jar (1069ms)
downloading http://repo1.maven.org/maven2/org/scala-tools/testing/test-interface/0.5/test-interface-0.5.jar ...
        [SUCCESSFUL ] org.scala-tools.testing#test-interface;0.5!test-interface.jar (175ms)
downloading http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar ...
        [SUCCESSFUL ] org.apache.ivy#ivy;2.1.0!ivy.jar (2584ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/compile_2.7.7/0.7.4/jars/compile_2.7.7.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#compile_2.7.7;0.7.4!compile_2.7.7.jar (798ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/launcher-interface/0.7.4/jars/launcher-interface.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#launcher-interface;0.7.4!launcher-interface.jar (306ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/interface/0.7.4/jars/interface.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#interface;0.7.4!interface.jar (457ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/ivy_2.7.7/0.7.4/jars/ivy_2.7.7.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#ivy_2.7.7;0.7.4!ivy_2.7.7.jar (767ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/io_2.7.7/0.7.4/jars/io_2.7.7.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#io_2.7.7;0.7.4!io_2.7.7.jar (460ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/classpath_2.7.7/0.7.4/jars/classpath_2.7.7.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#classpath_2.7.7;0.7.4!classpath_2.7.7.jar (306ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/compiler-interface/0.7.4/jars/compiler-interface-bin.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#compiler-interface;0.7.4!compiler-interface-bin.jar (313ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/compiler-interface/0.7.4/jars/compiler-interface-src.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#compiler-interface;0.7.4!compiler-interface-src.jar (305ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/control_2.7.7/0.7.4/jars/control_2.7.7.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#control_2.7.7;0.7.4!control_2.7.7.jar (303ms)
downloading http://repo1.maven.org/maven2/jline/jline/0.9.94/jline-0.9.94.jar ...
        [SUCCESSFUL ] jline#jline;0.9.94!jline.jar (907ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/precompiled-2.8.0.rc2_2.8.0.RC2/0.7.4/jars/compiler-interface-bin.jar ...
        [SUCCESSFUL ] org.scala-tools.sbt#precompiled-2.8.0.rc2_2.8.0.RC2;0.7.4!compiler-interface-bin.jar (608ms)
:: retrieving :: org.scala-tools.sbt#boot-app
        confs: [default]
        15 artifacts copied, 0 already retrieved (4096kB/27ms)
[info] Recompiling plugin definition...
[info]    Source analysis: 1 new/modified, 0 indirectly invalidated, 0 removed.
[info] 
[info] Updating plugins...
[info] downloading http://maven.twttr.com/com/twitter/standard-project/0.7.5/standard-project-0.7.5.jar ...
[info]  [SUCCESSFUL ] com.twitter#standard-project;0.7.5!standard-project.jar (539ms)
[info] downloading http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar ...
[info]  [SUCCESSFUL ] ivysvn#ivysvn;2.1.0!ivysvn.jar (4057ms)
[info] :: retrieving :: plugin-definition#plugin-definition_2.7.7 [sync]
[info]  confs: [compile, runtime, test, provided, system, optional, sources, javadoc]
[info]  2 artifacts copied, 0 already retrieved (2996kB/21ms)
[success] Plugins updated successfully.
[info] 
[info]  Extracted source plugin ./lib_managed/scala_2.7.7/standard-project-0.7.5.jar ...
[info] Recompiling plugin...
[info]    Source analysis: 13 new/modified, 0 indirectly invalidated, 0 removed.
[info] Recompiling project definition...
[info]    Source analysis: 1 new/modified, 0 indirectly invalidated, 0 removed.
[info] Standard project rules 0.7.5 loaded (2010-09-21).
[info] Building project kestrel 1.2.2 against Scala 2.7.7
[info]    using KestrelProject with sbt 0.7.4 and Scala 2.7.7
[info] 
[info] == clean-thrift ==
[info] == clean-thrift ==
[info] 
[info] == clean-dist ==
[info] == clean-dist ==
[info] 
[info] == clean ==
[info] == clean ==
[success] Successful.
[info] 
[info] Total time: 0 s, completed Sep 23, 2010 11:05:13 AM
[info] 
[info] == update ==
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/slf4j/slf4j-jdk14/1.5.2/slf4j-jdk14-1.5.2.jar ...
[info]  [SUCCESSFUL ] org.slf4j#slf4j-jdk14;1.5.2!slf4j-jdk14.jar (1059ms)
[info] downloading http://www.lag.net/nest/com/twitter/xrayspecs/1.0.7/xrayspecs-1.0.7.jar ...
[info]  [SUCCESSFUL ] com.twitter#xrayspecs;1.0.7!xrayspecs.jar (579ms)
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/scala-tools/testing/specs/1.6.2.1/specs-1.6.2.1.jar ...
[info]  [SUCCESSFUL ] org.scala-tools.testing#specs;1.6.2.1!specs.jar (9830ms)
[info] downloading http://www.lag.net/nest/com/twitter/twitteractors/1.0.0/twitteractors-1.0.0.jar ...
[info]  [SUCCESSFUL ] com.twitter#twitteractors;1.0.0!twitteractors.jar (1528ms)
[info] downloading http://scala-tools.org/repo-releases/net/lag/naggati/0.7.3/naggati-0.7.3.jar ...
[info]  [SUCCESSFUL ] net.lag#naggati;0.7.3!naggati.jar (7984ms)
[info] downloading http://scala-tools.org/repo-releases/net/lag/configgy/1.5.3/configgy-1.5.3.jar ...
[info]  [SUCCESSFUL ] net.lag#configgy;1.5.3!configgy.jar (668ms)
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/mina/mina-core/2.0.0-M6/mina-core-2.0.0-M6.jar ...
[info]  [SUCCESSFUL ] org.apache.mina#mina-core;2.0.0-M6!mina-core.jar(bundle) (5540ms)
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/slf4j/slf4j-api/1.5.2/slf4j-api-1.5.2.jar ...
[info]  [SUCCESSFUL ] org.slf4j#slf4j-api;1.5.2!slf4j-api.jar (350ms)
[info] downloading http://www.lag.net/repo/org/scala-tools/vscaladoc/1.1-md-3/vscaladoc-1.1-md-3.jar ...
[info]  [SUCCESSFUL ] org.scala-tools#vscaladoc;1.1-md-3!vscaladoc.jar (1505ms)
[info] :: retrieving :: net.lag#kestrel [sync]
[info]  confs: [compile, runtime, test, provided, system, optional, sources, javadoc]
[info]  9 artifacts copied, 0 already retrieved (4300kB/26ms)
[info] == update ==
[success] Successful.
[info] 
[info] Total time: 167 s, completed Sep 23, 2010 11:08:00 AM
[info] 
[info] == compile-thrift-ruby ==
[info] == compile-thrift-ruby ==
[info] 
[info] == compile-thrift-java ==
[info] == compile-thrift-java ==
[info] 
[info] == compile ==
[info]   Source analysis: 13 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling main sources...
[info] Compilation successful.
[info]   Post-analysis: 139 classes.
[info] == compile ==
[info] 
[info] == copy-test-resources ==
[info] == copy-test-resources ==
[info] 
[info] == test-compile ==
[info]   Source analysis: 11 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling test sources...
[info] Compilation successful.
[info]   Post-analysis: 592 classes.
[info] == test-compile ==
[info] 
[info] == copy-resources ==
[info] == copy-resources ==
[info] 
[info] == write-build-properties ==
[info] == write-build-properties ==
[info] 
[info] == copy-scripts ==
[info] == copy-scripts ==
[info] 
[info] == test-start ==
[info] == test-start ==
[info] 
[info] == net.lag.kestrel.PersistentQueueSpec ==
[info] PersistentQueueSpec
[info] PersistentQueue should
[info]   + add and remove one item
[info]   + resist adding an item that's too large
[info]   + flush all items
[info]   + rotate journals
[info]   + rotate journals with an open transaction
[info]   + recover the journal after a restart
[info]   + honor max_age
[info]   + allow max_journal_size and max_memory_size to be overridden per queue
[info]   + drop into read-behind mode
[info]     + on insert
[info]     + on startup
[info]     + during journal processing, then return to ordinary times
[info]   + handle timeout reads
[info]   + correctly interleave transactions in the journal
[info]   + recover a journal with open transactions
[info]   + recreate the journal file when it gets too big
[info]   + don't recreate the journal file if the queue itself is still huge
[info]   + report an age of zero on an empty queue
[info] PersistentQueue with no journal should
[info]   + create no journal
[info]   + lose all data after being destroyed
[info] PersistentQueue with item/size limit should
[info]   + honor max_items
[info]   + honor max_size
[info]   + drop older items when discard_old_when_full is set
[info] == net.lag.kestrel.PersistentQueueSpec ==
[info] 
[info] == net.lag.kestrel.QueueCollectionSpec ==
[info] QueueCollectionSpec
[info] QueueCollection should
[info]   + create a queue
[info]   + load from journal
[info]   + queue hit/miss tracking
[info]   + proactively load existing queue files
[info]   + ignore partially rolled queue files
[info]   + delete a queue when asked
[info]   + fanout queues
[info]     + generate on the fly
[info]     + preload existing
[info]     + delete on the fly
[info]   + move expired items from one queue to another
[info] == net.lag.kestrel.QueueCollectionSpec ==
[info] 
[info] == net.lag.kestrel.memcache.MemCacheCodecSpec ==
[info] MemCacheCodecSpec
[info] Memcache Decoder should
[info]   + 'get' request chunked various ways
[info]   + 'set' request chunked various ways
[info]   + 'quit' exits the session
[info] == net.lag.kestrel.memcache.MemCacheCodecSpec ==
[info] 
[info] == net.lag.kestrel.JournalSpec ==
[info] JournalSpec
[info] Journal should
[info]   + walk
[info]   + recover from corruption
[info] == net.lag.kestrel.JournalSpec ==
[info] 
[info] == net.lag.kestrel.ServerSpec ==
[info] ServerSpec
[info] Server should
[info]   + configure per-queue
[info]   + set and get one entry
[info]   + set with expiry
[info]   + set and get binary data
[info]   + commit a transactional get
[info]   + abort a transactional get
[info]   + auto-rollback a transaction on disconnect
[info]   + auto-commit cycles of transactional gets
[info]   + age
[info]   + peek
[info]   + rotate logs
[info]   + collect stats
[info]   + return a valid response for an unknown command
[info]   + disconnect and reconnect correctly
[info]   + flush expired items
[info] == net.lag.kestrel.ServerSpec ==
[info] 
[info] == test-complete ==
[info] == test-complete ==
[info] 
[info] == <anonymous> ==
[info] == <anonymous> ==
[info] 
[info] == package ==
[info] Packaging ./target/kestrel-1.2.2.jar ...
[info] Packaging complete.
[info] == package ==
[info] 
[info] == make-pom ==
[info] Wrote /usr/local/kestrel/target/kestrel-1.2.2.pom
[info] == make-pom ==
[info] 
[info] == test-finish ==
[info] Passed: : Total 54, Failed 0, Errors 0, Passed 54, Skipped 0
[info]  
[info] All tests PASSED.
[info] == test-finish ==
[info] 
[info] == test-cleanup ==
[info] == test-cleanup ==
[info] 
[info] == package-dist ==
[info] Packaging ./dist/kestrel-1.2.2.zip ...
[info] Packaging complete.
[info] == package-dist ==
[success] Successful.
[info] 
[info] Total time: 18 s, completed Sep 23, 2010 11:08:18 AM
[info] 
[info] Total session time: 201 s, completed Sep 23, 2010 11:08:18 AM
[success] Build completed successfully.
#

これでbuildできました。

とりあえず

とりあえず起動させるだけならjava -jarで直接実行するだけ大丈夫です。

java -jar /usr/local/kestrel/dist/kestrel-1.2.2/kestrel-1.2.2.jar &

または./dist/kestrel-1.2.2/scripts配下にあるkestrel.shを実行してやります。
(※環境によって若干修正必要あり)
kestrel.shで実行した場合は下記のようなプロセスになりました。

$ ps aux | grep java
root      4103  0.0  0.4 2471452 115380 ?      Sl   12:59   0:00 /root/jdk1.6.0_06/bin/java -server -verbosegc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Xmx2048m -Xms1024m -XX:NewSize=256m -jar /usr/local/kestrel/dist/kestrel-1.2.2/kestrel-1.2.2.jar
$

動作確認はデフォルトのポートとして22133ポートが起動するのでtelnetでアクセスします。

$ telnet localhost 22133
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
STATS
STAT uptime 10769
STAT time 1285225152
STAT version 1.2.2
STAT curr_items 0
STAT total_items 0
STAT bytes 0
STAT curr_connections 1
STAT total_connections 1
STAT cmd_get 0
STAT cmd_set 0
STAT cmd_peek 0
STAT get_hits 0
STAT get_misses 0
STAT bytes_read 6
STAT bytes_written 0
END
quit
Connection closed by foreign host.
$ 

最後に

ケストレルっていったら「イエス、ケストレル」だろjk