kestrelをインストールする
kestrelはメッセージキューの一種でtwitterでも利用されているらしい。
たまたまインストールする機会があったので作業メモ。
環境
OS: CentOS 5.5 (64bit)
jdk: 1.6.0
kestrel: http://github.com/robey/kestrel
git
kestrelのbuildはsbtってのを使うんだけど(後述)、
build処理の中でgitを呼び出してるらしく、gitがインストされていなとエラーになりました。
まぁkestrelのソース自体githubにあるのでこの機会に入れてしまいます。
# yum install git.x86_64
例によってCentOSのデフォのレポジトリでは入ってないかもしれない。
その時は、DAG(rpmforge)のレポジトリを追加もしくはソースからインストールする。
sbt
sbtはscalaのbuildツール。makeコマンドみたいなもん。
googlecodeからダウンロードできる。
インストールってゆうかコマンド作るって感じ。そのまま実行してもまぁいい。
$ wget http://simple-build-tool.googlecode.com/files/sbt-launch-0.7.4.jar $ vi sbt $ $ cat sbt java -Xmx512M -jar `dirname $0`/sbt-launch-0.7.4.jar "$@" $ $chmod u+x sbt $
パスは適宜通す。
めんどくさい場合は
$ java -Xmx512M -jar sbt-launch-0.7.4.jar foo bar
みたく直接コマンドを作っても良い。
kestre
ソースコードはgithubにあるのでそこからダウンロード。
どうせなんでgitコマンドを使う。
インストールディレクトリは/usr/local配下とする。
# cd /usr/local/
# git clone git://github.com/robey/kestrel.git
# cd kestrel/
# sbt clean update package-dist
Getting Scala 2.7.7 ...
downloading http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.7.7/scala-compiler-2.7.7.jar ...
[SUCCESSFUL ] org.scala-lang#scala-compiler;2.7.7!scala-compiler.jar (16737ms)
downloading http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.7.7/scala-library-2.7.7.jar ...
[SUCCESSFUL ] org.scala-lang#scala-library;2.7.7!scala-library.jar (9260ms)
:: retrieving :: org.scala-tools.sbt#boot-scala
confs: [default]
2 artifacts copied, 0 already retrieved (9911kB/45ms)
Getting org.scala-tools.sbt sbt_2.7.7 0.7.4 ...
downloading http://databinder.net/repo/org.scala-tools.sbt/sbt_2.7.7/0.7.4/jars/sbt_2.7.7.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#sbt_2.7.7;0.7.4!sbt_2.7.7.jar (2382ms)
downloading http://repo1.maven.org/maven2/com/jcraft/jsch/0.1.31/jsch-0.1.31.jar ...
[SUCCESSFUL ] com.jcraft#jsch;0.1.31!jsch.jar (1069ms)
downloading http://repo1.maven.org/maven2/org/scala-tools/testing/test-interface/0.5/test-interface-0.5.jar ...
[SUCCESSFUL ] org.scala-tools.testing#test-interface;0.5!test-interface.jar (175ms)
downloading http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar ...
[SUCCESSFUL ] org.apache.ivy#ivy;2.1.0!ivy.jar (2584ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/compile_2.7.7/0.7.4/jars/compile_2.7.7.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#compile_2.7.7;0.7.4!compile_2.7.7.jar (798ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/launcher-interface/0.7.4/jars/launcher-interface.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#launcher-interface;0.7.4!launcher-interface.jar (306ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/interface/0.7.4/jars/interface.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#interface;0.7.4!interface.jar (457ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/ivy_2.7.7/0.7.4/jars/ivy_2.7.7.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#ivy_2.7.7;0.7.4!ivy_2.7.7.jar (767ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/io_2.7.7/0.7.4/jars/io_2.7.7.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#io_2.7.7;0.7.4!io_2.7.7.jar (460ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/classpath_2.7.7/0.7.4/jars/classpath_2.7.7.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#classpath_2.7.7;0.7.4!classpath_2.7.7.jar (306ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/compiler-interface/0.7.4/jars/compiler-interface-bin.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#compiler-interface;0.7.4!compiler-interface-bin.jar (313ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/compiler-interface/0.7.4/jars/compiler-interface-src.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#compiler-interface;0.7.4!compiler-interface-src.jar (305ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/control_2.7.7/0.7.4/jars/control_2.7.7.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#control_2.7.7;0.7.4!control_2.7.7.jar (303ms)
downloading http://repo1.maven.org/maven2/jline/jline/0.9.94/jline-0.9.94.jar ...
[SUCCESSFUL ] jline#jline;0.9.94!jline.jar (907ms)
downloading http://databinder.net/repo/org.scala-tools.sbt/precompiled-2.8.0.rc2_2.8.0.RC2/0.7.4/jars/compiler-interface-bin.jar ...
[SUCCESSFUL ] org.scala-tools.sbt#precompiled-2.8.0.rc2_2.8.0.RC2;0.7.4!compiler-interface-bin.jar (608ms)
:: retrieving :: org.scala-tools.sbt#boot-app
confs: [default]
15 artifacts copied, 0 already retrieved (4096kB/27ms)
[info] Recompiling plugin definition...
[info] Source analysis: 1 new/modified, 0 indirectly invalidated, 0 removed.
[info]
[info] Updating plugins...
[info] downloading http://maven.twttr.com/com/twitter/standard-project/0.7.5/standard-project-0.7.5.jar ...
[info] [SUCCESSFUL ] com.twitter#standard-project;0.7.5!standard-project.jar (539ms)
[info] downloading http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar ...
[info] [SUCCESSFUL ] ivysvn#ivysvn;2.1.0!ivysvn.jar (4057ms)
[info] :: retrieving :: plugin-definition#plugin-definition_2.7.7 [sync]
[info] confs: [compile, runtime, test, provided, system, optional, sources, javadoc]
[info] 2 artifacts copied, 0 already retrieved (2996kB/21ms)
[success] Plugins updated successfully.
[info]
[info] Extracted source plugin ./lib_managed/scala_2.7.7/standard-project-0.7.5.jar ...
[info] Recompiling plugin...
[info] Source analysis: 13 new/modified, 0 indirectly invalidated, 0 removed.
[info] Recompiling project definition...
[info] Source analysis: 1 new/modified, 0 indirectly invalidated, 0 removed.
[info] Standard project rules 0.7.5 loaded (2010-09-21).
[info] Building project kestrel 1.2.2 against Scala 2.7.7
[info] using KestrelProject with sbt 0.7.4 and Scala 2.7.7
[info]
[info] == clean-thrift ==
[info] == clean-thrift ==
[info]
[info] == clean-dist ==
[info] == clean-dist ==
[info]
[info] == clean ==
[info] == clean ==
[success] Successful.
[info]
[info] Total time: 0 s, completed Sep 23, 2010 11:05:13 AM
[info]
[info] == update ==
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/slf4j/slf4j-jdk14/1.5.2/slf4j-jdk14-1.5.2.jar ...
[info] [SUCCESSFUL ] org.slf4j#slf4j-jdk14;1.5.2!slf4j-jdk14.jar (1059ms)
[info] downloading http://www.lag.net/nest/com/twitter/xrayspecs/1.0.7/xrayspecs-1.0.7.jar ...
[info] [SUCCESSFUL ] com.twitter#xrayspecs;1.0.7!xrayspecs.jar (579ms)
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/scala-tools/testing/specs/1.6.2.1/specs-1.6.2.1.jar ...
[info] [SUCCESSFUL ] org.scala-tools.testing#specs;1.6.2.1!specs.jar (9830ms)
[info] downloading http://www.lag.net/nest/com/twitter/twitteractors/1.0.0/twitteractors-1.0.0.jar ...
[info] [SUCCESSFUL ] com.twitter#twitteractors;1.0.0!twitteractors.jar (1528ms)
[info] downloading http://scala-tools.org/repo-releases/net/lag/naggati/0.7.3/naggati-0.7.3.jar ...
[info] [SUCCESSFUL ] net.lag#naggati;0.7.3!naggati.jar (7984ms)
[info] downloading http://scala-tools.org/repo-releases/net/lag/configgy/1.5.3/configgy-1.5.3.jar ...
[info] [SUCCESSFUL ] net.lag#configgy;1.5.3!configgy.jar (668ms)
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/mina/mina-core/2.0.0-M6/mina-core-2.0.0-M6.jar ...
[info] [SUCCESSFUL ] org.apache.mina#mina-core;2.0.0-M6!mina-core.jar(bundle) (5540ms)
[info] downloading http://mirrors.ibiblio.org/pub/mirrors/maven2/org/slf4j/slf4j-api/1.5.2/slf4j-api-1.5.2.jar ...
[info] [SUCCESSFUL ] org.slf4j#slf4j-api;1.5.2!slf4j-api.jar (350ms)
[info] downloading http://www.lag.net/repo/org/scala-tools/vscaladoc/1.1-md-3/vscaladoc-1.1-md-3.jar ...
[info] [SUCCESSFUL ] org.scala-tools#vscaladoc;1.1-md-3!vscaladoc.jar (1505ms)
[info] :: retrieving :: net.lag#kestrel [sync]
[info] confs: [compile, runtime, test, provided, system, optional, sources, javadoc]
[info] 9 artifacts copied, 0 already retrieved (4300kB/26ms)
[info] == update ==
[success] Successful.
[info]
[info] Total time: 167 s, completed Sep 23, 2010 11:08:00 AM
[info]
[info] == compile-thrift-ruby ==
[info] == compile-thrift-ruby ==
[info]
[info] == compile-thrift-java ==
[info] == compile-thrift-java ==
[info]
[info] == compile ==
[info] Source analysis: 13 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling main sources...
[info] Compilation successful.
[info] Post-analysis: 139 classes.
[info] == compile ==
[info]
[info] == copy-test-resources ==
[info] == copy-test-resources ==
[info]
[info] == test-compile ==
[info] Source analysis: 11 new/modified, 0 indirectly invalidated, 0 removed.
[info] Compiling test sources...
[info] Compilation successful.
[info] Post-analysis: 592 classes.
[info] == test-compile ==
[info]
[info] == copy-resources ==
[info] == copy-resources ==
[info]
[info] == write-build-properties ==
[info] == write-build-properties ==
[info]
[info] == copy-scripts ==
[info] == copy-scripts ==
[info]
[info] == test-start ==
[info] == test-start ==
[info]
[info] == net.lag.kestrel.PersistentQueueSpec ==
[info] PersistentQueueSpec
[info] PersistentQueue should
[info] + add and remove one item
[info] + resist adding an item that's too large
[info] + flush all items
[info] + rotate journals
[info] + rotate journals with an open transaction
[info] + recover the journal after a restart
[info] + honor max_age
[info] + allow max_journal_size and max_memory_size to be overridden per queue
[info] + drop into read-behind mode
[info] + on insert
[info] + on startup
[info] + during journal processing, then return to ordinary times
[info] + handle timeout reads
[info] + correctly interleave transactions in the journal
[info] + recover a journal with open transactions
[info] + recreate the journal file when it gets too big
[info] + don't recreate the journal file if the queue itself is still huge
[info] + report an age of zero on an empty queue
[info] PersistentQueue with no journal should
[info] + create no journal
[info] + lose all data after being destroyed
[info] PersistentQueue with item/size limit should
[info] + honor max_items
[info] + honor max_size
[info] + drop older items when discard_old_when_full is set
[info] == net.lag.kestrel.PersistentQueueSpec ==
[info]
[info] == net.lag.kestrel.QueueCollectionSpec ==
[info] QueueCollectionSpec
[info] QueueCollection should
[info] + create a queue
[info] + load from journal
[info] + queue hit/miss tracking
[info] + proactively load existing queue files
[info] + ignore partially rolled queue files
[info] + delete a queue when asked
[info] + fanout queues
[info] + generate on the fly
[info] + preload existing
[info] + delete on the fly
[info] + move expired items from one queue to another
[info] == net.lag.kestrel.QueueCollectionSpec ==
[info]
[info] == net.lag.kestrel.memcache.MemCacheCodecSpec ==
[info] MemCacheCodecSpec
[info] Memcache Decoder should
[info] + 'get' request chunked various ways
[info] + 'set' request chunked various ways
[info] + 'quit' exits the session
[info] == net.lag.kestrel.memcache.MemCacheCodecSpec ==
[info]
[info] == net.lag.kestrel.JournalSpec ==
[info] JournalSpec
[info] Journal should
[info] + walk
[info] + recover from corruption
[info] == net.lag.kestrel.JournalSpec ==
[info]
[info] == net.lag.kestrel.ServerSpec ==
[info] ServerSpec
[info] Server should
[info] + configure per-queue
[info] + set and get one entry
[info] + set with expiry
[info] + set and get binary data
[info] + commit a transactional get
[info] + abort a transactional get
[info] + auto-rollback a transaction on disconnect
[info] + auto-commit cycles of transactional gets
[info] + age
[info] + peek
[info] + rotate logs
[info] + collect stats
[info] + return a valid response for an unknown command
[info] + disconnect and reconnect correctly
[info] + flush expired items
[info] == net.lag.kestrel.ServerSpec ==
[info]
[info] == test-complete ==
[info] == test-complete ==
[info]
[info] == <anonymous> ==
[info] == <anonymous> ==
[info]
[info] == package ==
[info] Packaging ./target/kestrel-1.2.2.jar ...
[info] Packaging complete.
[info] == package ==
[info]
[info] == make-pom ==
[info] Wrote /usr/local/kestrel/target/kestrel-1.2.2.pom
[info] == make-pom ==
[info]
[info] == test-finish ==
[info] Passed: : Total 54, Failed 0, Errors 0, Passed 54, Skipped 0
[info]
[info] All tests PASSED.
[info] == test-finish ==
[info]
[info] == test-cleanup ==
[info] == test-cleanup ==
[info]
[info] == package-dist ==
[info] Packaging ./dist/kestrel-1.2.2.zip ...
[info] Packaging complete.
[info] == package-dist ==
[success] Successful.
[info]
[info] Total time: 18 s, completed Sep 23, 2010 11:08:18 AM
[info]
[info] Total session time: 201 s, completed Sep 23, 2010 11:08:18 AM
[success] Build completed successfully.
#これでbuildできました。
とりあえず
とりあえず起動させるだけならjava -jarで直接実行するだけ大丈夫です。
java -jar /usr/local/kestrel/dist/kestrel-1.2.2/kestrel-1.2.2.jar &
または./dist/kestrel-1.2.2/scripts配下にあるkestrel.shを実行してやります。
(※環境によって若干修正必要あり)
kestrel.shで実行した場合は下記のようなプロセスになりました。
$ ps aux | grep java root 4103 0.0 0.4 2471452 115380 ? Sl 12:59 0:00 /root/jdk1.6.0_06/bin/java -server -verbosegc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Xmx2048m -Xms1024m -XX:NewSize=256m -jar /usr/local/kestrel/dist/kestrel-1.2.2/kestrel-1.2.2.jar $
動作確認はデフォルトのポートとして22133ポートが起動するのでtelnetでアクセスします。
$ telnet localhost 22133 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. STATS STAT uptime 10769 STAT time 1285225152 STAT version 1.2.2 STAT curr_items 0 STAT total_items 0 STAT bytes 0 STAT curr_connections 1 STAT total_connections 1 STAT cmd_get 0 STAT cmd_set 0 STAT cmd_peek 0 STAT get_hits 0 STAT get_misses 0 STAT bytes_read 6 STAT bytes_written 0 END quit Connection closed by foreign host. $
最後に
ケストレルっていったら「イエス、ケストレル」だろjk
