====== fluentd + elasticsearch + Kibana3 によるログ収集・可視化 ====== 温度計などのセンサデータを、MA-E3xx により収集し、[[http://www.fluentd.org/|fluentd]] により [[http://www.elasticsearch.org/|elasticsearch]] に集約し、\\ [[http://www.elasticsearch.org/overview/kibana/|Kibana3]] により可視化を行ってみます。\\ 今流行りの BuzzWord, "IoT" っていう感じでしょうか。 構成イメージはこのようになります。 {{:mae3xx_tips:setup_fluentd:mae3xx_fluentd_elasticsearch_00.svg?800|}} (画像ファイルは公式サイトより流用) シリアルポートのデータを、TCP 変換してサーバに接続するというのは簡単だとは思いますが、\\ 3G 回線が切れたときなど、通信ができないときのデータ欠損が避けらず、オススメできません。\\ シリアルポート通信はリモート側で終端し、この仕組みでデータを送信するのが今時の仕組みではないでしょうか。 \\ ===== 参考 ===== ==== 書籍 ==== 最高の参考書が発売されました。 [[http://gihyo.jp/book/2014/978-4-7741-6983-5|{{http://image.gihyo.co.jp/assets/images/cover/2014/thumb/TH160_9784774169835.jpg|サーバ／インフラエンジニア養成読本 ログ収集〜可視化編}}]] ==== Web ==== * [[http://repeatedly.github.io/ja/2014/07/fluentd-and-log-forwarding-patterns/|Fluentdとログ収集のパターン]] (Go ahead! - Memoization for Everything) * [[http://knowledge.sakura.ad.jp/tech/1336/|柔軟なログ収集を可能にする「fluentd」入門]] (さくらのナレッジ) * [[http://www.iij.ad.jp/company/development/tech/activities/dc_data/|研究開発用データセンターの空調制御効率化を目指したデータ収集]] (IIJ) * [[http://www.slideshare.net/hsugipon/fluentddc-15092795|Fluentdによる研究開発用DCのデータ収集 (slideshare)]] (上記 IIJ のプレゼン資料, Vaisala 使ってます) * [[http://blog.zoncoen.net/blog/2013/11/11/logging-system-with-fluentd-elasticsearch-kibana3/|Fluentd + ElasticSearch + Kibana3で簡単に様々なログを可視化・解析する]] (404 Engineer Logs) * [[http://blog.johtani.info/blog/2013/06/19/introduction-kibana3/|Kibana3というのもありまして]] (@johtaniの日記 2nd) * [[https://www.digitalocean.com/community/tutorials/elasticsearch-fluentd-and-kibana-open-source-log-search-and-visualization|Elasticsearch, Fluentd, and Kibana: Open Source Log Search and Visualization]] (DigitalOcean) * [[https://www.sraoss.co.jp/event_seminar/2014/20140325-fluentd.pdf|ログ収集基盤 fluentdのご紹介(PDF)]] (SRA OSS) * [[http://www.publickey1.jp/blog/14/fluentdkubernetes.html|FluentdがKubernetesの標準ログ収集ツールとして採用]] (Publickey) \\ ===== 準備 ===== MA-E3xx 側には、fluentd をインストールする必要があります。 残念ながら、Ubuntu14.04 用の公式パッケージは用意されておりません。\\ パッケージでインストールしておかないと後々管理ができなくなりますので、 Mario FetkaさんのPPA [[https://launchpad.net/~mario-fetka/+archive/fluentd]] \\ からパッケージを作成し、インストールすることにします。 {{:mae3xx_tips:setup_fluentd:ppa_image.png|https://launchpad.net/~mario-fetka/+archive/fluentd}} \\ パッケージ作成にあたっては、下記環境が前提となります。 * 開発環境一式 (build-essential) をインストール済みであること。((参照: [[mae3xx_devel:build_essential_setup:start]])) * パッケージ作成用のパッケージ(devscripts)をインストール済みであること。((参照: [[mae3xx_tips:setup_softether:start]])) * Ruby 開発環境 (ruby-dev) をインストール済みであること。 ※ fluentd を組み込んだファームウェアを、"[[download_software:mae3xx:mae3xx_fluentd:start]]" に置いてあります。 \\ ==== Ruby 開発環境のセットアップ ==== Ruby2.0 を使用したい(([[mae3xx_tips:test_fib_benchmark:start]] の結果の通り、Ruby1.9系より高速なためです。))ので、ruby2.0, ruby2.0-dev をインストールします。 user1@plum:~$ sudo apt-get install ruby2.0 ruby2.0-dev Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: libjs-jquery libruby1.9.1 libruby2.0 libyaml-0-2 ruby ruby1.9.1 rubygems-integration Suggested packages: javascript-common ri ruby-dev ruby1.9.1-examples ri1.9.1 graphviz ruby1.9.1-dev ruby-switch bundler The following NEW packages will be installed: libjs-jquery libruby1.9.1 libruby2.0 libyaml-0-2 ruby ruby1.9.1 ruby2.0 ruby2.0-dev rubygems-integration 0 upgraded, 9 newly installed, 0 to remove and 2 not upgraded. Need to get 6089 kB of archives. After this operation, 25.5 MB of additional disk space will be used. Do you want to continue? [Y/n] Get:1 http://ports.ubuntu.com/ubuntu-ports/ trusty/main libyaml-0-2 armhf 0.1.4-3ubuntu3 [40.4 kB] Get:2 http://ports.ubuntu.com/ubuntu-ports/ trusty/main libjs-jquery all 1.7.2+dfsg-2ubuntu1 [78.8 kB] Get:3 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby all 1:1.9.3.4 [5334 B] Get:4 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby1.9.1 armhf 1.9.3.484-2ubuntu1 [35.5 kB] Get:5 http://ports.ubuntu.com/ubuntu-ports/ trusty/main libruby1.9.1 armhf 1.9.3.484-2ubuntu1 [2458 kB] Get:6 http://ports.ubuntu.com/ubuntu-ports/ trusty/main libruby2.0 armhf 2.0.0.484-1ubuntu2 [2599 kB] Get:7 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby2.0 armhf 2.0.0.484-1ubuntu2 [66.3 kB] Get:8 http://ports.ubuntu.com/ubuntu-ports/ trusty/main rubygems-integration all 1.5 [5340 B] Get:9 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby2.0-dev armhf 2.0.0.484-1ubuntu2 [800 kB] Fetched 6089 kB in 10s (590 kB/s) Selecting previously unselected package libyaml-0-2:armhf. (Reading database ... 21471 files and directories currently installed.) Preparing to unpack .../libyaml-0-2_0.1.4-3ubuntu3_armhf.deb ... Unpacking libyaml-0-2:armhf (0.1.4-3ubuntu3) ... Selecting previously unselected package libjs-jquery. Preparing to unpack .../libjs-jquery_1.7.2+dfsg-2ubuntu1_all.deb ... Unpacking libjs-jquery (1.7.2+dfsg-2ubuntu1) ... Selecting previously unselected package ruby. Preparing to unpack .../ruby_1%3a1.9.3.4_all.deb ... Unpacking ruby (1:1.9.3.4) ... Selecting previously unselected package ruby1.9.1. Preparing to unpack .../ruby1.9.1_1.9.3.484-2ubuntu1_armhf.deb ... Unpacking ruby1.9.1 (1.9.3.484-2ubuntu1) ... Selecting previously unselected package libruby1.9.1. Preparing to unpack .../libruby1.9.1_1.9.3.484-2ubuntu1_armhf.deb ... Unpacking libruby1.9.1 (1.9.3.484-2ubuntu1) ... Selecting previously unselected package libruby2.0:armhf. Preparing to unpack .../libruby2.0_2.0.0.484-1ubuntu2_armhf.deb ... Unpacking libruby2.0:armhf (2.0.0.484-1ubuntu2) ... Selecting previously unselected package ruby2.0. Preparing to unpack .../ruby2.0_2.0.0.484-1ubuntu2_armhf.deb ... Unpacking ruby2.0 (2.0.0.484-1ubuntu2) ... Selecting previously unselected package rubygems-integration. Preparing to unpack .../rubygems-integration_1.5_all.deb ... Unpacking rubygems-integration (1.5) ... Selecting previously unselected package ruby2.0-dev:armhf. Preparing to unpack .../ruby2.0-dev_2.0.0.484-1ubuntu2_armhf.deb ... Unpacking ruby2.0-dev:armhf (2.0.0.484-1ubuntu2) ... Setting up libyaml-0-2:armhf (0.1.4-3ubuntu3) ... Setting up libjs-jquery (1.7.2+dfsg-2ubuntu1) ... Setting up ruby2.0-dev:armhf (2.0.0.484-1ubuntu2) ... Setting up ruby (1:1.9.3.4) ... Setting up ruby1.9.1 (1.9.3.484-2ubuntu1) ... Setting up libruby1.9.1 (1.9.3.484-2ubuntu1) ... Setting up rubygems-integration (1.5) ... Setting up ruby2.0 (2.0.0.484-1ubuntu2) ... Setting up libruby2.0:armhf (2.0.0.484-1ubuntu2) ... Processing triggers for libc-bin (2.19-0ubuntu6) ... localepurge: Disk space freed in /usr/share/locale: 0 KiB localepurge: Disk space freed in /usr/share/man: 0 KiB Total disk space freed by localepurge: 0 KiB user1@plum:~$ \\ 上のとおり、ruby1.9.1 も一緒にインストールされてしまいます。 user1@plum:~$ ruby -v ruby 1.9.3p484 (2013-11-22 revision 43786) [arm-linux-eabihf] user1@plum:~$ \\ 単に "ruby" として実行すると、Ruby1.9 系がデフォルトとなっているため、Ruby2.0 系をデフォルトにする設定をします。\\ 参考: [[http://mhag.hatenablog.com/entry/2014/07/04/151415|´・ω・)＜ときどき書くよ - Ubuntu 14.04 LTS に Fluentd をインストールする]] user1@plum:~$ sudo update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 0 update-alternatives: using /usr/bin/ruby1.9.1 to provide /usr/bin/ruby (ruby) in auto mode user1@plum:~$ sudo update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby2.0 1 update-alternatives: using /usr/bin/ruby2.0 to provide /usr/bin/ruby (ruby) in auto mode user1@plum:~$ ruby -v ruby 2.0.0p384 (2014-01-12) [arm-linux-gnueabihf] user1@plum:~$ \\ Gem も同様に 2.0 系をデフォルトに変更します。 user1@plum:~$ sudo update-alternatives --install /usr/bin/gem gem /usr/bin/gem1.9.1 0 update-alternatives: using /usr/bin/gem1.9.1 to provide /usr/bin/gem (gem) in auto mode user1@plum:~$ sudo update-alternatives --install /usr/bin/gem gem /usr/bin/gem2.0 1 update-alternatives: using /usr/bin/gem2.0 to provide /usr/bin/gem (gem) in auto mode user1@plum:~$ gem -v 2.0.14 user1@plum:~$ \\ ===== パッケージの作成 ===== ==== PPA の追加 ==== PPA を追加します。 user1@plum:~$ sudo add-apt-repository ppa:mario-fetka/fluentd [sudo] password for user1: Fluentd is an open source data collector designed for processing data streams. More info: https://launchpad.net/~mario-fetka/+archive/ubuntu/fluentd Press [ENTER] to continue or ctrl-c to cancel adding it gpg: keyring `/tmp/tmpavrdryx3/secring.gpg' created gpg: keyring `/tmp/tmpavrdryx3/pubring.gpg' created gpg: requesting key F1AA2FAE from hkp server keyserver.ubuntu.com gpg: /tmp/tmpavrdryx3/trustdb.gpg: trustdb created gpg: key F1AA2FAE: public key "Launchpad PPA for Mario Fetka" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) OK user1@plum:~$ \\ バイナリパッケージが準備されていないものがありますので、PPA のソースパッケージの項を有効にします。 user1@plum:~$ sudo nano -w /etc/apt/sources.list.d/mario-fetka-fluentd-trusty.list deb http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu trusty main # deb-src http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu trusty main <-- "deb-src" の前のコメントを外して保存します。 \\ 追加された PPA のパッケージ情報を取得するため、apt-get update によりパッケージ情報を更新します。 user1@plum:~$ sudo apt-get update 〜〜〜 略 〜〜〜 Get:1 http://ppa.launchpad.net trusty/main Sources [12.6 kB] 〜〜〜 略 〜〜〜 Fetched 12.6 kB in 7s (1590 B/s) Reading package lists... Done user1@plum:~$ \\ ==== パッケージの確認 ==== "fluentd" 関連のパッケージを検索してみます。 user1@plum:~$ sudo apt-cache search fluentd ruby-fluentd - Fluentd event collector ruby-fluent-mixin-config-placeholders - Configuration syntax extension mixin for fluentd plugin ruby-fluent-mixin-plaintextformatter - Text formatter mixin module to create fluentd plugin ruby-fluent-mixin-rewrite-tag-name - Fluentd mixin plugin to provides placeholder function for rewriting tag for your any plugins as like fluent-plugin-rewrite-tag-filter. It will let you get easy to implement tag placeholder for your own plugins. ruby-fluent-plugin-config-expander - Fluentd plugin to serve some DSL directives in configuration ruby-fluent-plugin-event-tail - fluentd input plugin for reading [tag, time, record] messages from a file ruby-fluent-plugin-file-sprintf - sprintf output file plugin for Fluentd. ruby-fluent-plugin-flatten - Fluentd plugin to extract values for nested key paths and re-emit them as flat tag/record pairs. ruby-fluent-plugin-geoip - Fluentd Output plugin to add information about geographical location of IP addresses with Maxmind GeoIP databases. ruby-fluent-plugin-glusterfs - Fluentd plugin for GlusterFS ruby-fluent-plugin-grep - fluentd plugin to grep messages ruby-fluent-plugin-in-udp-event - Event driven udp input plugin for fluentd ruby-fluent-plugin-record-reformer - Fluentd plugin to add or replace fields of a event record ruby-fluent-plugin-reemit - Fluentd plugin to re-emit messages avoiding infinity match loop ruby-fluent-plugin-rewrite - Fluentd plugin to rewrite tags/values along with pattern matching and re-emit them. ruby-fluent-plugin-rewrite-tag-filter - Fluentd Output filter plugin. It has designed to rewrite tag like mod_rewrite. Re-emmit a record with rewrited tag when a value matches/unmatches with the regular expression. Also you can change a tag from apache log by domain, status-code(ex. 500 error), user-agent, request-uri, regex-backreference and so on with regular expression. user1@plum:~$ \\ ==== パッケージのビルド ==== Fluentd 本体は "ruby-fluentd" ですが、下記パッケージに依存しています。 * ruby-iobuffer * ruby-cool.io 両方共 ARM 用バイナリパッケージが準備されていませんので、ソースパッケージをダウンロードしてビルドを行います。 \\ === ruby-iobuffer のビルド === user1@plum:~$ sudo apt-get source ruby-iobuffer Reading package lists... Done Building dependency tree Reading state information... Done Need to get 9663 B of source archives. Get:1 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-iobuffer 1.1.2-2ppa1~trusty1 (tar) [6394 B] Get:2 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-iobuffer 1.1.2-2ppa1~trusty1 (diff) [1949 B] Get:3 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-iobuffer 1.1.2-2ppa1~trusty1 (dsc) [1320 B] Fetched 9663 B in 1s (8186 B/s) gpgv: Signature made Fri Apr 18 19:24:59 2014 JST using DSA key ID 0F0C7602 gpgv: Can't check signature: public key not found dpkg-source: warning: failed to verify signature on ./ruby-iobuffer_1.1.2-2ppa1~trusty1.dsc dpkg-source: info: extracting ruby-iobuffer in ruby-iobuffer-1.1.2 dpkg-source: info: unpacking ruby-iobuffer_1.1.2.orig.tar.gz dpkg-source: info: unpacking ruby-iobuffer_1.1.2-2ppa1~trusty1.debian.tar.gz user1@plum:~$ user1@plum:~$ cd ruby-iobuffer-1.1.2 user1@plum:~/ruby-iobuffer-1.1.2$ sudo debuild -us -uc -b dpkg-buildpackage -rfakeroot -D -us -uc -b dpkg-buildpackage: warning: using a gain-root-command while being root dpkg-buildpackage: source package ruby-iobuffer dpkg-buildpackage: source version 1.1.2-2ppa1~trusty1 dpkg-buildpackage: source distribution trusty dpkg-buildpackage: source changed by Mario Fetka dpkg-source --before-build ruby-iobuffer-1.1.2 dpkg-buildpackage: host architecture armhf dpkg-source: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file dpkg-checkbuilddeps: Unmet build dependencies: debhelper (>= 7.0.50~) gem2deb (>= 0.3.0~) rake rake-compiler ruby-rspec dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting dpkg-buildpackage: warning: (Use -d flag to override.) debuild: fatal error at line 1364: dpkg-buildpackage -rfakeroot -D -us -uc -b failed user1@plum:~/ruby-iobuffer-1.1.2$ \\ 上の通り、いくつかのパッケージがビルドには必要となります。提示されたパッケージをインストールします。\\ "fakeroot" も必要となりますのでインストールしておきます。 user1@plum:~/ruby-iobuffer-1.1.2$ sudo apt-get install debhelper gem2deb rake rake-compiler ruby-rspec fakeroot Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: bsdmainutils dh-apparmor gettext gettext-base groff-base intltool-debian libasprintf0c2 libcroco3 libfakeroot libunistring0 man-db po-debconf ruby-diff-lcs ruby-rspec-core ruby-rspec-expectations ruby-rspec-mocks ruby1.9.1-dev Suggested packages: wamerican wordlist whois vacation dh-make apparmor-easyprof gettext-doc groff www-browser libmail-box-perl Recommended packages: libasprintf-dev libgettextpo-dev libmail-sendmail-perl The following NEW packages will be installed: bsdmainutils debhelper dh-apparmor fakeroot gem2deb gettext gettext-base groff-base intltool-debian libasprintf0c2 libcroco3 libfakeroot libunistring0 man-db po-debconf rake rake-compiler ruby-diff-lcs ruby-rspec ruby-rspec-core ruby-rspec-expectations ruby-rspec-mocks ruby1.9.1-dev 0 upgraded, 23 newly installed, 0 to remove and 2 not upgraded. Need to get 5179 kB of archives. After this operation, 16.4 MB of additional disk space will be used. Do you want to continue? [Y/n] Get:1 http://ports.ubuntu.com/ubuntu-ports/ trusty-updates/main libasprintf0c2 armhf 0.18.3.1-1ubuntu3 [6488 B] Get:2 http://ports.ubuntu.com/ubuntu-ports/ trusty/main groff-base armhf 1.22.2-5 [934 kB] Get:3 http://ports.ubuntu.com/ubuntu-ports/ trusty/main bsdmainutils armhf 9.0.5ubuntu1 [195 kB] Get:4 http://ports.ubuntu.com/ubuntu-ports/ trusty/main man-db armhf 2.6.7.1-1 [832 kB] Get:5 http://ports.ubuntu.com/ubuntu-ports/ trusty/main libcroco3 armhf 0.6.8-2ubuntu1 [69.1 kB] Get:6 http://ports.ubuntu.com/ubuntu-ports/ trusty/main libunistring0 armhf 0.9.3-5ubuntu3 [251 kB] Get:7 http://ports.ubuntu.com/ubuntu-ports/ trusty-updates/main gettext-base armhf 0.18.3.1-1ubuntu3 [45.8 kB] Get:8 http://ports.ubuntu.com/ubuntu-ports/ trusty-updates/main gettext armhf 0.18.3.1-1ubuntu3 [779 kB] Get:9 http://ports.ubuntu.com/ubuntu-ports/ trusty/main intltool-debian all 0.35.0+20060710.1 [31.6 kB] Get:10 http://ports.ubuntu.com/ubuntu-ports/ trusty/main po-debconf all 1.0.16+nmu2ubuntu1 [210 kB] Get:11 http://ports.ubuntu.com/ubuntu-ports/ trusty/main dh-apparmor all 2.8.95~2430-0ubuntu5 [11.3 kB] Get:12 http://ports.ubuntu.com/ubuntu-ports/ trusty/main debhelper all 9.20131227ubuntu1 [604 kB] Get:13 http://ports.ubuntu.com/ubuntu-ports/ trusty/main libfakeroot armhf 1.20-3ubuntu2 [21.1 kB] Get:14 http://ports.ubuntu.com/ubuntu-ports/ trusty/main fakeroot armhf 1.20-3ubuntu2 [55.2 kB] Get:15 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby1.9.1-dev armhf 1.9.3.484-2ubuntu1 [784 kB] Get:16 http://ports.ubuntu.com/ubuntu-ports/ trusty/main gem2deb all 0.6.1 [41.8 kB] Get:17 http://ports.ubuntu.com/ubuntu-ports/ trusty/main rake all 10.0.4-1 [49.0 kB] Get:18 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe rake-compiler all 0.9.2-1 [17.4 kB] Get:19 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby-diff-lcs all 1.2.4-1 [26.9 kB] Get:20 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby-rspec-expectations all 2.14.2-1 [57.9 kB] Get:21 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby-rspec-mocks all 2.14.3-1 [52.7 kB] Get:22 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby-rspec-core all 2.14.7-2 [97.0 kB] Get:23 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby-rspec all 2.14.1-1 [4996 B] Fetched 5179 kB in 14s (353 kB/s) Preconfiguring packages ... Selecting previously unselected package libasprintf0c2:armhf. (Reading database ... 23420 files and directories currently installed.) Preparing to unpack .../libasprintf0c2_0.18.3.1-1ubuntu3_armhf.deb ... Unpacking libasprintf0c2:armhf (0.18.3.1-1ubuntu3) ... Selecting previously unselected package groff-base. Preparing to unpack .../groff-base_1.22.2-5_armhf.deb ... Unpacking groff-base (1.22.2-5) ... Selecting previously unselected package bsdmainutils. Preparing to unpack .../bsdmainutils_9.0.5ubuntu1_armhf.deb ... Unpacking bsdmainutils (9.0.5ubuntu1) ... Selecting previously unselected package man-db. Preparing to unpack .../man-db_2.6.7.1-1_armhf.deb ... Unpacking man-db (2.6.7.1-1) ... Selecting previously unselected package libcroco3:armhf. Preparing to unpack .../libcroco3_0.6.8-2ubuntu1_armhf.deb ... Unpacking libcroco3:armhf (0.6.8-2ubuntu1) ... Selecting previously unselected package libunistring0:armhf. Preparing to unpack .../libunistring0_0.9.3-5ubuntu3_armhf.deb ... Unpacking libunistring0:armhf (0.9.3-5ubuntu3) ... Selecting previously unselected package gettext-base. Preparing to unpack .../gettext-base_0.18.3.1-1ubuntu3_armhf.deb ... Unpacking gettext-base (0.18.3.1-1ubuntu3) ... Selecting previously unselected package gettext. Preparing to unpack .../gettext_0.18.3.1-1ubuntu3_armhf.deb ... Unpacking gettext (0.18.3.1-1ubuntu3) ... Selecting previously unselected package intltool-debian. Preparing to unpack .../intltool-debian_0.35.0+20060710.1_all.deb ... Unpacking intltool-debian (0.35.0+20060710.1) ... Selecting previously unselected package po-debconf. Preparing to unpack .../po-debconf_1.0.16+nmu2ubuntu1_all.deb ... Unpacking po-debconf (1.0.16+nmu2ubuntu1) ... Selecting previously unselected package dh-apparmor. Preparing to unpack .../dh-apparmor_2.8.95~2430-0ubuntu5_all.deb ... Unpacking dh-apparmor (2.8.95~2430-0ubuntu5) ... Selecting previously unselected package debhelper. Preparing to unpack .../debhelper_9.20131227ubuntu1_all.deb ... Unpacking debhelper (9.20131227ubuntu1) ... Selecting previously unselected package libfakeroot:armhf. Preparing to unpack .../libfakeroot_1.20-3ubuntu2_armhf.deb ... Unpacking libfakeroot:armhf (1.20-3ubuntu2) ... Selecting previously unselected package fakeroot. Preparing to unpack .../fakeroot_1.20-3ubuntu2_armhf.deb ... Unpacking fakeroot (1.20-3ubuntu2) ... Selecting previously unselected package ruby1.9.1-dev. Preparing to unpack .../ruby1.9.1-dev_1.9.3.484-2ubuntu1_armhf.deb ... Unpacking ruby1.9.1-dev (1.9.3.484-2ubuntu1) ... Selecting previously unselected package gem2deb. Preparing to unpack .../archives/gem2deb_0.6.1_all.deb ... Unpacking gem2deb (0.6.1) ... Selecting previously unselected package rake. Preparing to unpack .../archives/rake_10.0.4-1_all.deb ... Unpacking rake (10.0.4-1) ... Selecting previously unselected package rake-compiler. Preparing to unpack .../rake-compiler_0.9.2-1_all.deb ... Unpacking rake-compiler (0.9.2-1) ... Selecting previously unselected package ruby-diff-lcs. Preparing to unpack .../ruby-diff-lcs_1.2.4-1_all.deb ... Unpacking ruby-diff-lcs (1.2.4-1) ... Selecting previously unselected package ruby-rspec-expectations. Preparing to unpack .../ruby-rspec-expectations_2.14.2-1_all.deb ... Unpacking ruby-rspec-expectations (2.14.2-1) ... Selecting previously unselected package ruby-rspec-mocks. Preparing to unpack .../ruby-rspec-mocks_2.14.3-1_all.deb ... Unpacking ruby-rspec-mocks (2.14.3-1) ... Selecting previously unselected package ruby-rspec-core. Preparing to unpack .../ruby-rspec-core_2.14.7-2_all.deb ... Unpacking ruby-rspec-core (2.14.7-2) ... Selecting previously unselected package ruby-rspec. Preparing to unpack .../ruby-rspec_2.14.1-1_all.deb ... Unpacking ruby-rspec (2.14.1-1) ... Processing triggers for mime-support (3.54ubuntu1) ... Setting up libasprintf0c2:armhf (0.18.3.1-1ubuntu3) ... Setting up groff-base (1.22.2-5) ... Setting up bsdmainutils (9.0.5ubuntu1) ... update-alternatives: using /usr/bin/bsd-write to provide /usr/bin/write (write) in auto mode update-alternatives: warning: skip creation of /usr/share/man/man1/write.1.gz because associated file /usr/share/man/man1/bsd-write.1.gz (of link group write) doesn't exist update-alternatives: using /usr/bin/bsd-from to provide /usr/bin/from (from) in auto mode update-alternatives: warning: skip creation of /usr/share/man/man1/from.1.gz because associated file /usr/share/man/man1/bsd-from.1.gz (of link group from) doesn't exist Setting up man-db (2.6.7.1-1) ... Building database of manual pages ... Setting up libcroco3:armhf (0.6.8-2ubuntu1) ... Setting up libunistring0:armhf (0.9.3-5ubuntu3) ... Setting up gettext-base (0.18.3.1-1ubuntu3) ... Setting up gettext (0.18.3.1-1ubuntu3) ... Setting up intltool-debian (0.35.0+20060710.1) ... Setting up po-debconf (1.0.16+nmu2ubuntu1) ... Setting up dh-apparmor (2.8.95~2430-0ubuntu5) ... Setting up debhelper (9.20131227ubuntu1) ... Setting up libfakeroot:armhf (1.20-3ubuntu2) ... Setting up fakeroot (1.20-3ubuntu2) ... update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode update-alternatives: warning: skip creation of /usr/share/man/man1/fakeroot.1.gz because associated file /usr/share/man/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist update-alternatives: warning: skip creation of /usr/share/man/man1/faked.1.gz because associated file /usr/share/man/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist update-alternatives: warning: skip creation of /usr/share/man/es/man1/fakeroot.1.gz because associated file /usr/share/man/es/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist update-alternatives: warning: skip creation of /usr/share/man/es/man1/faked.1.gz because associated file /usr/share/man/es/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist update-alternatives: warning: skip creation of /usr/share/man/fr/man1/fakeroot.1.gz because associated file /usr/share/man/fr/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist update-alternatives: warning: skip creation of /usr/share/man/fr/man1/faked.1.gz because associated file /usr/share/man/fr/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist update-alternatives: warning: skip creation of /usr/share/man/sv/man1/fakeroot.1.gz because associated file /usr/share/man/sv/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist update-alternatives: warning: skip creation of /usr/share/man/sv/man1/faked.1.gz because associated file /usr/share/man/sv/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist Setting up ruby1.9.1-dev (1.9.3.484-2ubuntu1) ... Setting up gem2deb (0.6.1) ... Setting up rake (10.0.4-1) ... Setting up rake-compiler (0.9.2-1) ... Setting up ruby-diff-lcs (1.2.4-1) ... Setting up ruby-rspec-expectations (2.14.2-1) ... Setting up ruby-rspec-mocks (2.14.3-1) ... Setting up ruby-rspec-core (2.14.7-2) ... Setting up ruby-rspec (2.14.1-1) ... Processing triggers for libc-bin (2.19-0ubuntu6) ... localepurge: Disk space freed in /usr/share/locale: 0 KiB localepurge: Disk space freed in /usr/share/man: 0 KiB Total disk space freed by localepurge: 0 KiB user1@plum:~/ruby-iobuffer-1.1.2$ \\ 必要なパッケージを追加しましたので、再び debuild コマンドを実行します。 user1@plum:~/ruby-iobuffer-1.1.2$ sudo debuild -us -uc -b dpkg-buildpackage -rfakeroot -D -us -uc -b dpkg-buildpackage: warning: using a gain-root-command while being root dpkg-buildpackage: source package ruby-iobuffer dpkg-buildpackage: source version 1.1.2-2ppa1~trusty1 dpkg-buildpackage: source distribution trusty dpkg-buildpackage: source changed by Mario Fetka dpkg-source --before-build ruby-iobuffer-1.1.2 dpkg-buildpackage: host architecture armhf dpkg-source: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file fakeroot debian/rules clean dh clean --buildsystem=ruby --with ruby dh_testdir -O--buildsystem=ruby dh_auto_clean -O--buildsystem=ruby /usr/lib/ruby/vendor_ruby/gem2deb.rb:17: warning: setting Encoding.default_external 〜〜〜 略 〜〜〜 dpkg-gencontrol: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file dpkg-gencontrol: warning: File::FcntlLock not available; using flock which is not NFS-safe dpkg-gencontrol: warning: package ruby-iobuffer: unused substitution variable ${misc:Pre-Depends} dh_md5sums -O--buildsystem=ruby dh_builddeb -O--buildsystem=ruby dpkg-deb: warning: 'debian/ruby-iobuffer/DEBIAN/control' contains user-defined field 'Ruby-Versions' dpkg-deb: warning: ignoring 1 warning about the control file(s) dpkg-deb: building package `ruby-iobuffer' in `../ruby-iobuffer_1.1.2-2ppa1~trusty1_armhf.deb'. dpkg-genchanges -b >../ruby-iobuffer_1.1.2-2ppa1~trusty1_armhf.changes dpkg-genchanges: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file dpkg-genchanges: binary-only upload - not including any source code dpkg-source --after-build ruby-iobuffer-1.1.2 dpkg-source: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file dpkg-buildpackage: binary only upload (no source included) user1@plum:~/ruby-iobuffer-1.1.2$ \\ パッケージが作成されました。 user1@plum:~/ruby-iobuffer-1.1.2$ cd .. user1@plum:~$ ls -l ruby-iobuffer*deb -rw-r--r-- 1 root root 7634 Jul 25 14:13 ruby-iobuffer_1.1.2-2ppa1~trusty1_armhf.deb user1@plum:~$ \\ ビルドされたパッケージをインストールしておきます。 user1@plum:~$ sudo dpkg -i ruby-iobuffer_1.1.2-2ppa1~trusty1_armhf.deb Selecting previously unselected package ruby-iobuffer. (Reading database ... 25155 files and directories currently installed.) Preparing to unpack ruby-iobuffer_1.1.2-2ppa1~trusty1_armhf.deb ... Unpacking ruby-iobuffer (1.1.2-2ppa1~trusty1) ... Setting up ruby-iobuffer (1.1.2-2ppa1~trusty1) ... user1@plum:~$ \\ === ruby-cool.io のビルド === "ruby-iobuffer" と同様にして、ビルドを行います。 user1@plum:~$ sudo apt-get source ruby-cool.io Reading package lists... Done Building dependency tree Reading state information... Done Need to get 98.1 kB of source archives. Get:1 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-cool.io 1.1.1-2ppa1~trusty1 (tar) [94.6 kB] Get:2 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-cool.io 1.1.1-2ppa1~trusty1 (diff) [2214 B] Get:3 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-cool.io 1.1.1-2ppa1~trusty1 (dsc) [1317 B] Fetched 98.1 kB in 1s (49.9 kB/s) gpgv: Signature made Fri Apr 18 19:27:58 2014 JST using DSA key ID 0F0C7602 gpgv: Can't check signature: public key not found dpkg-source: warning: failed to verify signature on ./ruby-cool.io_1.1.1-2ppa1~trusty1.dsc dpkg-source: info: extracting ruby-cool.io in ruby-cool.io-1.1.1 dpkg-source: info: unpacking ruby-cool.io_1.1.1.orig.tar.gz dpkg-source: info: unpacking ruby-cool.io_1.1.1-2ppa1~trusty1.debian.tar.gz user1@plum:~$ user1@plum:~$ cd ruby-cool.io-1.1.1/ user1@plum:~/ruby-cool.io-1.1.1$ sudo debuild -us -uc -b dpkg-buildpackage -rfakeroot -D -us -uc -b dpkg-buildpackage: warning: using a gain-root-command while being root dpkg-buildpackage: source package ruby-cool.io dpkg-buildpackage: source version 1.1.1-2ppa1~trusty1 dpkg-buildpackage: source distribution trusty dpkg-buildpackage: source changed by Mario Fetka dpkg-source --before-build ruby-cool.io-1.1.1 dpkg-buildpackage: host architecture armhf dpkg-source: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file fakeroot debian/rules clean dh clean --buildsystem=ruby --with ruby dh_testdir -O--buildsystem=ruby 〜〜〜 略 〜〜〜 dpkg-deb: warning: 'debian/ruby-cool.io/DEBIAN/control' contains user-defined field 'Ruby-Versions' dpkg-deb: warning: ignoring 1 warning about the control file(s) dpkg-deb: building package `ruby-cool.io' in `../ruby-cool.io_1.1.1-2ppa1~trusty1_armhf.deb'. dpkg-genchanges -b >../ruby-cool.io_1.1.1-2ppa1~trusty1_armhf.changes dpkg-genchanges: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file dpkg-genchanges: binary-only upload - not including any source code dpkg-source --after-build ruby-cool.io-1.1.1 dpkg-source: warning: unknown information field 'Dm-Upload-Allowed' in input data in general section of control info file dpkg-buildpackage: binary only upload (no source included) user1@plum:~/ruby-cool.io-1.1.1$ \\ こちらもパッケージが作成されました。 user1@plum:~/ruby-cool.io-1.1.1$ cd .. user1@plum:~$ ls -l ruby-cool.io*deb -rw-r--r-- 1 root root 47100 Jul 25 14:37 ruby-cool.io_1.1.1-2ppa1~trusty1_armhf.deb user1@plum:~$ \\ 同様に、ビルドされたパッケージをインストールしておきます。 user1@plum:~$ sudo dpkg -i ruby-cool.io_1.1.1-2ppa1~trusty1_armhf.deb Selecting previously unselected package ruby-cool.io. (Reading database ... 25167 files and directories currently installed.) Preparing to unpack ruby-cool.io_1.1.1-2ppa1~trusty1_armhf.deb ... Unpacking ruby-cool.io (1.1.1-2ppa1~trusty1) ... Setting up ruby-cool.io (1.1.1-2ppa1~trusty1) ... user1@plum:~$ \\ ===== ruby-fluentd のインストール ===== 以上で、"ruby-fluentd" が依存する(バイナリパッケージが用意されていない)パッケージのビルド・インストールができましたので、\\ ruby-fluentd をインストールします。 user1@plum:~$ sudo apt-get install ruby-fluentd Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: libhttp-parser2.1 libjemalloc1 ruby-http-parser.rb ruby-json ruby-msgpack ruby-sigdump ruby-yajl Suggested packages: ruby-http-parser.rb-doc The following NEW packages will be installed: libhttp-parser2.1 libjemalloc1 ruby-fluentd ruby-http-parser.rb ruby-json ruby-msgpack ruby-sigdump ruby-yajl 0 upgraded, 8 newly installed, 0 to remove and 2 not upgraded. Need to get 254 kB of archives. After this operation, 1533 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-sigdump all 0.2.2-2ppa1~trusty1 [3784 B] Get:2 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe libhttp-parser2.1 armhf 2.1-2 [17.3 kB] Get:3 http://ppa.launchpad.net/mario-fetka/fluentd/ubuntu/ trusty/main ruby-fluentd all 0.10.45-2ppa1~trusty1 [50.9 kB] Get:4 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe libjemalloc1 armhf 3.5.1-2 [70.6 kB] Get:5 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe ruby-msgpack armhf 0.5.5-2 [25.6 kB] Get:6 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ruby-json armhf 1.8.0-1build1 [44.0 kB] Get:7 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe ruby-yajl armhf 1.2.0-1 [32.0 kB] Get:8 http://ports.ubuntu.com/ubuntu-ports/ trusty/universe ruby-http-parser.rb armhf 0.6.0~beta.2-1 [10.1 kB] Fetched 254 kB in 3s (83.4 kB/s) Selecting previously unselected package libhttp-parser2.1:armhf. (Reading database ... 25193 files and directories currently installed.) Preparing to unpack .../libhttp-parser2.1_2.1-2_armhf.deb ... Unpacking libhttp-parser2.1:armhf (2.1-2) ... Selecting previously unselected package libjemalloc1. Preparing to unpack .../libjemalloc1_3.5.1-2_armhf.deb ... Unpacking libjemalloc1 (3.5.1-2) ... Selecting previously unselected package ruby-msgpack. Preparing to unpack .../ruby-msgpack_0.5.5-2_armhf.deb ... Unpacking ruby-msgpack (0.5.5-2) ... Selecting previously unselected package ruby-json. Preparing to unpack .../ruby-json_1.8.0-1build1_armhf.deb ... Unpacking ruby-json (1.8.0-1build1) ... Selecting previously unselected package ruby-yajl. Preparing to unpack .../ruby-yajl_1.2.0-1_armhf.deb ... Unpacking ruby-yajl (1.2.0-1) ... Selecting previously unselected package ruby-http-parser.rb. Preparing to unpack .../ruby-http-parser.rb_0.6.0~beta.2-1_armhf.deb ... Unpacking ruby-http-parser.rb (0.6.0~beta.2-1) ... Selecting previously unselected package ruby-sigdump. Preparing to unpack .../ruby-sigdump_0.2.2-2ppa1~trusty1_all.deb ... Unpacking ruby-sigdump (0.2.2-2ppa1~trusty1) ... Selecting previously unselected package ruby-fluentd. Preparing to unpack .../ruby-fluentd_0.10.45-2ppa1~trusty1_all.deb ... Unpacking ruby-fluentd (0.10.45-2ppa1~trusty1) ... Processing triggers for ureadahead (0.100.0-16) ... Setting up libhttp-parser2.1:armhf (2.1-2) ... Setting up libjemalloc1 (3.5.1-2) ... Setting up ruby-msgpack (0.5.5-2) ... Setting up ruby-json (1.8.0-1build1) ... Setting up ruby-yajl (1.2.0-1) ... Setting up ruby-http-parser.rb (0.6.0~beta.2-1) ... Setting up ruby-sigdump (0.2.2-2ppa1~trusty1) ... Setting up ruby-fluentd (0.10.45-2ppa1~trusty1) ... Creating system group: fluent Adding group `fluent' (GID 112) ... Done. Creating user fluent in group fluent * Starting fluent fluent 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:214:rescue in main_process: unexpected error error="No such file or directory - /etc/fluent/fluent.conf" 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:276:in `read' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:276:in `read_config' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:88:in `block in start' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:198:in `call' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:198:in `main_process' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:173:in `block in supervise' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:172:in `fork' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:172:in `supervise' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/supervisor.rb:87:in `start' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/vendor_ruby/fluent/command/fluentd.rb:168:in `' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in `require' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in `require' 2014-07-25 14:43:38 +0900 [error]: fluent/supervisor.rb:172:fork: /usr/bin/fluentd:6:in `
' [fail] Processing triggers for libc-bin (2.19-0ubuntu6) ... Processing triggers for ureadahead (0.100.0-16) ... localepurge: Disk space freed in /usr/share/locale: 0 KiB localepurge: Disk space freed in /usr/share/man: 0 KiB Total disk space freed by localepurge: 0 KiB user1@plum:~$ \\ セットアップ後、自動で起動処理が行われてしまいました。\\ 設定ファイル /etc/fluent/fluent.conf が無いので起動は失敗したようです。 \\ ==== 動作テスト ==== 簡単な設定ファイルを作り、基本的な動作確認をしてみます。 user1@plum:~$ sudo mkdir /etc/fluent user1@plum:~$ sudo nano -w /etc/fluent/fluent.conf type forward type file path /tmp/fluent_debug.txt 標準入力から入ってきたものの中で、タグが "debug.*" にマッチしたものを /tmp/fluent_debug.txt へ出力する設定です。 \\ fluentd を起動します。 user1@plum:~$ sudo service ruby-fluentd start * Starting fluent fluent [ OK ] user1@plum:~$ \\ 単純なデータを入力してみます。タグは "debug.test" にします。 user1@plum:~$ echo '{"key" : "value"}' | fluent-cat debug.test \\ 出力されたか確認してみます。 user1@plum:~$ ls -l /tmp/fluent_debug* -rw-r--r-- 1 fluent fluent 109 Jul 25 15:09 /tmp/fluent_debug.txt.20140725.b4fefe4bdbaa9b437 user1@plum:~$ cat /tmp/fluent_debug.txt.20140725.b4fefe4bdbaa9b437 2014-07-25T15:09:41+09:00 debug.test {"key":"value"} 処理が行われたのが確認できました。 \\ ===== サーバの設定 ===== ==== コンテナの作成 ==== クライアント(MA-E3xx)側の Fluentd はとりあえずセットアップできましたので、データを蓄積・可視化するサーバ側もセットアップします。\\ 今回は、サーバには LXC((docker による方法も、のちほど試そうと考えています)) によるコンテナ型仮想環境を準備します。\\ Fluentd の公式パッケージがサポートしているのが Ubuntu 12.04LTS (Precise) なので、Ubuntu 12.04LTS の環境を作成します。\\ 参照: [[http://docs.fluentd.org/articles/install-by-deb]] user1@ubuntu:~$ sudo lxc-create -t ubuntu -n precise64 -- -r precise Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ... Installing packages in template: ssh,vim,language-pack-en,language-pack-ja Downloading ubuntu precise minimal ... I: Retrieving Release I: Retrieving Release.gpg I: Checking Release signature I: Valid Release signature (key id 630239CC130E1A7FD81A27B140976EAF437D05B5) 〜〜〜 略 〜〜〜 Generating locales... en_US.UTF-8... done Generation complete. Creating SSH2 RSA key; this may take some time ... Creating SSH2 DSA key; this may take some time ... Creating SSH2 ECDSA key; this may take some time ... Current default time zone: 'Asia/Tokyo' Local time is now: Fri Aug 1 16:20:33 JST 2014. Universal Time is now: Fri Aug 1 07:20:33 UTC 2014. ## # The default user is 'ubuntu' with password 'ubuntu'! # Use the 'sudo' command to run tasks as root in the container. ## user1@ubuntu:~$ \\ ==== コンテナの起動 / 設定 ==== 作成したコンテナを起動します。\\ まだネットワークの設定を行っていませんので、フォアグランドで起動させます。 user1@ubuntu:~$ sudo lxc-start -n precise64 Ubuntu 12.04.4 LTS precise64 console precise64 login: <4>init: setvtrgb main process (272) terminated with status 1 Ubuntu 12.04.4 LTS precise64 console precise64 login: ubuntu Password: Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.13.0-32-generic x86_64) * Documentation: https://help.ubuntu.com/ The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. ubuntu@precise64:~$ ログインできました。 \\ 使用する環境に合わせて、 * ネットワーク設定 * ユーザアカウント追加 などを行っておきます。ubuntu ユーザも削除しておくほうがよいと思います。 \\ 設定が完了したら一旦コンテナを終了し、再度 -d オプション (daemonize the container) 付きで起動します。 user1@ubuntu:~$ sudo lxc-start -n precise64 -d \\ 今度は SSH でログインします。 user1@ubuntu:~$ ssh -l user1 192.168.103.192 user1@192.168.103.192's password: Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.13.0-32-generic x86_64) * Documentation: https://help.ubuntu.com/ Last login: Fri Aug 1 16:31:33 2014 user1@precise64:~$ \\ ==== td-agent のセットアップ ==== [[http://docs.fluentd.org/articles/install-by-deb|公式サイトの説明]] のとおり作業します。\\ ※ curl は初期状態ではインストールされていないため、apt-get でインストールが必要です。 user1@precise64:~$ curl -L http://toolbelt.treasuredata.com/sh/install-ubuntu-precise.sh | sh % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 555 100 555 0 0 3037 0 --:--:-- --:--:-- --:--:-- 92500 This script requires superuser access to install apt packages. You will be prompted for your password by sudo. [sudo] password for user1: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 934 100 934 0 0 4330 0 --:--:-- --:--:-- --:--:-- 130k OK Get:1 http://packages.treasuredata.com precise Release.gpg [198 B] Get:2 http://packages.treasuredata.com precise Release [2,377 B] Get:3 http://packages.treasuredata.com precise/contrib amd64 Packages [447 B] Get:4 http://packages.treasuredata.com precise/contrib i386 Packages [451 B] Ign http://packages.treasuredata.com precise/contrib TranslationIndex Hit http://archive.ubuntu.com precise Release.gpg Hit http://archive.ubuntu.com precise-updates Release.gpg Hit http://security.ubuntu.com precise-security Release.gpg Hit http://archive.ubuntu.com precise Release Hit http://security.ubuntu.com precise-security Release Hit http://archive.ubuntu.com precise-updates Release Hit http://security.ubuntu.com precise-security/main amd64 Packages Hit http://archive.ubuntu.com precise/main amd64 Packages Hit http://archive.ubuntu.com precise/restricted amd64 Packages Hit http://archive.ubuntu.com precise/universe amd64 Packages Hit http://archive.ubuntu.com precise/multiverse amd64 Packages Hit http://archive.ubuntu.com precise/main i386 Packages Hit http://archive.ubuntu.com precise/restricted i386 Packages Hit http://archive.ubuntu.com precise/universe i386 Packages Hit http://archive.ubuntu.com precise/multiverse i386 Packages Hit http://archive.ubuntu.com precise/main TranslationIndex Hit http://archive.ubuntu.com precise/multiverse TranslationIndex Hit http://security.ubuntu.com precise-security/restricted amd64 Packages Hit http://security.ubuntu.com precise-security/universe amd64 Packages Hit http://security.ubuntu.com precise-security/multiverse amd64 Packages Hit http://security.ubuntu.com precise-security/main i386 Packages Hit http://security.ubuntu.com precise-security/restricted i386 Packages Hit http://security.ubuntu.com precise-security/universe i386 Packages Hit http://security.ubuntu.com precise-security/multiverse i386 Packages Hit http://security.ubuntu.com precise-security/main TranslationIndex Hit http://security.ubuntu.com precise-security/multiverse TranslationIndex Hit http://security.ubuntu.com precise-security/restricted TranslationIndex Hit http://archive.ubuntu.com precise/restricted TranslationIndex Hit http://archive.ubuntu.com precise/universe TranslationIndex Hit http://archive.ubuntu.com precise-updates/main amd64 Packages Hit http://archive.ubuntu.com precise-updates/restricted amd64 Packages Hit http://archive.ubuntu.com precise-updates/universe amd64 Packages Hit http://archive.ubuntu.com precise-updates/multiverse amd64 Packages Hit http://archive.ubuntu.com precise-updates/main i386 Packages Ign http://packages.treasuredata.com precise/contrib Translation-en_US Hit http://security.ubuntu.com precise-security/universe TranslationIndex Ign http://packages.treasuredata.com precise/contrib Translation-en Hit http://archive.ubuntu.com precise-updates/restricted i386 Packages Hit http://archive.ubuntu.com precise-updates/universe i386 Packages Hit http://archive.ubuntu.com precise-updates/multiverse i386 Packages Hit http://archive.ubuntu.com precise-updates/main TranslationIndex Hit http://archive.ubuntu.com precise-updates/multiverse TranslationIndex Hit http://archive.ubuntu.com precise-updates/restricted TranslationIndex Hit http://archive.ubuntu.com precise-updates/universe TranslationIndex Hit http://archive.ubuntu.com precise/main Translation-en Hit http://security.ubuntu.com precise-security/main Translation-en Hit http://security.ubuntu.com precise-security/multiverse Translation-en Hit http://archive.ubuntu.com precise/multiverse Translation-en Hit http://archive.ubuntu.com precise/restricted Translation-en Hit http://security.ubuntu.com precise-security/restricted Translation-en Hit http://archive.ubuntu.com precise/universe Translation-en Hit http://archive.ubuntu.com precise-updates/main Translation-en Hit http://archive.ubuntu.com precise-updates/multiverse Translation-en Hit http://archive.ubuntu.com precise-updates/restricted Translation-en Hit http://archive.ubuntu.com precise-updates/universe Translation-en Hit http://security.ubuntu.com precise-security/universe Translation-en Fetched 3,473 B in 2s (1,383 B/s) Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: libcap2 libclass-isa-perl libgdbm3 libopts25 libssl0.9.8 libswitch-perl libxml2 libxslt1.1 libyaml-0-2 ntp perl perl-modules sgml-base xml-core Suggested packages: ntp-doc apparmor perl-doc libterm-readline-gnu-perl libterm-readline-perl-perl make libpod-plainer-perl sgml-base-doc debhelper The following NEW packages will be installed: libcap2 libclass-isa-perl libgdbm3 libopts25 libssl0.9.8 libswitch-perl libxml2 libxslt1.1 libyaml-0-2 ntp perl perl-modules sgml-base td-agent xml-core 0 upgraded, 15 newly installed, 0 to remove and 0 not upgraded. Need to get 71.3 MB of archives. After this operation, 136 MB of additional disk space will be used. Get:1 http://packages.treasuredata.com/precise/ precise/contrib td-agent amd64 1.1.20-1 [61.0 MB] Get:2 http://archive.ubuntu.com/ubuntu/ precise-updates/universe libssl0.9.8 amd64 0.9.8o-7ubuntu3.2 [868 kB] Get:3 http://archive.ubuntu.com/ubuntu/ precise/main libgdbm3 amd64 1.8.3-10 [35.3 kB] Get:4 http://archive.ubuntu.com/ubuntu/ precise-updates/main libxml2 amd64 2.7.8.dfsg-5.1ubuntu4.9 [673 kB] Get:5 http://archive.ubuntu.com/ubuntu/ precise/main libcap2 amd64 1:2.22-1ubuntu3 [12.0 kB] Get:6 http://archive.ubuntu.com/ubuntu/ precise-updates/main libxslt1.1 amd64 1.1.26-8ubuntu1.3 [167 kB] Get:7 http://archive.ubuntu.com/ubuntu/ precise-updates/main libyaml-0-2 amd64 0.1.4-2ubuntu0.12.04.3 [57.8 kB] Get:8 http://archive.ubuntu.com/ubuntu/ precise/main libopts25 amd64 1:5.12-0.1ubuntu1 [59.9 kB] Get:9 http://archive.ubuntu.com/ubuntu/ precise-updates/main ntp amd64 1:4.2.6.p3+dfsg-1ubuntu3.1 [612 kB] Get:10 http://archive.ubuntu.com/ubuntu/ precise/main sgml-base all 1.26+nmu1ubuntu1 [8,360 B] Get:11 http://archive.ubuntu.com/ubuntu/ precise/main xml-core all 0.13 [23.4 kB] Get:12 http://archive.ubuntu.com/ubuntu/ precise/main libclass-isa-perl all 0.36-3 [11.9 kB] Get:13 http://archive.ubuntu.com/ubuntu/ precise-updates/main perl-modules all 5.14.2-6ubuntu2.4 [3,389 kB] Get:14 http://archive.ubuntu.com/ubuntu/ precise-updates/main perl amd64 5.14.2-6ubuntu2.4 [4,411 kB] Get:15 http://archive.ubuntu.com/ubuntu/ precise/main libswitch-perl all 2.16-2 [19.2 kB] Fetched 71.3 MB in 6s (10.8 MB/s) Preconfiguring packages ... Selecting previously unselected package libssl0.9.8. (Reading database ... 12876 files and directories currently installed.) Unpacking libssl0.9.8 (from .../libssl0.9.8_0.9.8o-7ubuntu3.2_amd64.deb) ... Selecting previously unselected package libgdbm3. Unpacking libgdbm3 (from .../libgdbm3_1.8.3-10_amd64.deb) ... Selecting previously unselected package libxml2. Unpacking libxml2 (from .../libxml2_2.7.8.dfsg-5.1ubuntu4.9_amd64.deb) ... Selecting previously unselected package libcap2. Unpacking libcap2 (from .../libcap2_1%3a2.22-1ubuntu3_amd64.deb) ... Selecting previously unselected package libxslt1.1. Unpacking libxslt1.1 (from .../libxslt1.1_1.1.26-8ubuntu1.3_amd64.deb) ... Selecting previously unselected package libyaml-0-2. Unpacking libyaml-0-2 (from .../libyaml-0-2_0.1.4-2ubuntu0.12.04.3_amd64.deb) ... Selecting previously unselected package libopts25. Unpacking libopts25 (from .../libopts25_1%3a5.12-0.1ubuntu1_amd64.deb) ... Selecting previously unselected package ntp. Unpacking ntp (from .../ntp_1%3a4.2.6.p3+dfsg-1ubuntu3.1_amd64.deb) ... Selecting previously unselected package sgml-base. Unpacking sgml-base (from .../sgml-base_1.26+nmu1ubuntu1_all.deb) ... Selecting previously unselected package xml-core. Unpacking xml-core (from .../archives/xml-core_0.13_all.deb) ... Selecting previously unselected package libclass-isa-perl. Unpacking libclass-isa-perl (from .../libclass-isa-perl_0.36-3_all.deb) ... Selecting previously unselected package perl-modules. Unpacking perl-modules (from .../perl-modules_5.14.2-6ubuntu2.4_all.deb) ... Selecting previously unselected package perl. Unpacking perl (from .../perl_5.14.2-6ubuntu2.4_amd64.deb) ... Selecting previously unselected package libswitch-perl. Unpacking libswitch-perl (from .../libswitch-perl_2.16-2_all.deb) ... Selecting previously unselected package td-agent. Unpacking td-agent (from .../td-agent_1.1.20-1_amd64.deb) ... Processing triggers for ureadahead ... Setting up libssl0.9.8 (0.9.8o-7ubuntu3.2) ... Setting up libgdbm3 (1.8.3-10) ... Setting up libxml2 (2.7.8.dfsg-5.1ubuntu4.9) ... Setting up libcap2 (1:2.22-1ubuntu3) ... Setting up libxslt1.1 (1.1.26-8ubuntu1.3) ... Setting up libyaml-0-2 (0.1.4-2ubuntu0.12.04.3) ... Setting up libopts25 (1:5.12-0.1ubuntu1) ... Setting up ntp (1:4.2.6.p3+dfsg-1ubuntu3.1) ... * Starting NTP server ntpd [ OK ] Setting up sgml-base (1.26+nmu1ubuntu1) ... Setting up xml-core (0.13) ... Setting up libclass-isa-perl (0.36-3) ... Setting up td-agent (1.1.20-1) ... Adding system user `td-agent' (UID 104) ... Adding new group `td-agent' (GID 106) ... Adding new user `td-agent' (UID 104) with group `td-agent' ... Not creating home directory `/home/td-agent'. Installing default conffile /etc/td-agent/td-agent.conf ... * Starting td-agent td-agent [ OK ] Setting up libswitch-perl (2.16-2) ... Setting up perl-modules (5.14.2-6ubuntu2.4) ... Setting up perl (5.14.2-6ubuntu2.4) ... update-alternatives: using /usr/bin/prename to provide /usr/bin/rename (rename) in auto mode. Processing triggers for libc-bin ... ldconfig deferred processing now taking place user1@precise64:~$ \\ 起動しているか、確認します。 user1@precise64:~$ ps ax PID TTY STAT TIME COMMAND 1 ? Ss 0:00 /sbin/init 114 ? S 0:00 upstart-udev-bridge --daemon 145 ? Ss 0:00 /sbin/udevd --daemon 153 ? Sl 0:00 rsyslogd -c5 155 ? S 0:00 upstart-socket-bridge --daemon 203 ? Ss 0:00 /usr/sbin/sshd -D 231 ? Ss+ 0:00 /sbin/getty -8 38400 tty4 234 ? Ss+ 0:00 /sbin/getty -8 38400 tty2 235 ? Ss+ 0:00 /sbin/getty -8 38400 tty3 241 ? Ss 0:00 cron 268 ? Ss+ 0:00 /sbin/getty -8 38400 console 271 ? Ss+ 0:00 /sbin/getty -8 38400 tty1 303 ? Ss 0:00 sshd: user1 [priv] 315 ? S 0:00 sshd: user1@pts/0 316 pts/0 Ss 0:00 -bash 3609 ? Ss 0:00 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 103:105 3732 ? Sl 0:00 /usr/lib/fluent/ruby/bin/ruby /usr/sbin/td-agent --daemon /var/run/td-agent/td-agent.pid --log /var/log/td-agent/td-agent.log 3735 ? Sl 0:00 /usr/lib/fluent/ruby/bin/ruby /usr/sbin/td-agent --daemon /var/run/td-agent/td-agent.pid --log /var/log/td-agent/td-agent.log 3757 pts/0 R+ 0:00 ps ax user1@precise64:~$ \\ ==== Java のセットアップ ==== elasticsearch は JVM 上で動作しますので、Java をインストールしておきます。\\ 参照: [[http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html|Install Oracle Java 7 in Ubuntu via PPA Repository]] 準備 user1@precise64:~$ sudo apt-get install python-software-properties user1@precise64:~$ sudo add-apt-repository ppa:webupd8team/java You are about to add the following PPA to your system: Oracle Java (JDK) Installer (automatically downloads and installs Oracle JDK6 / JDK7 / JDK8). There are no actual Java files in this PPA. More info: - for Oracle Java 7: http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html - for Oracle Java 8: http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html Debian installation instructions: http://www.webupd8.org/2012/06/how-to-install-oracle-java-7-in-debian.html More info: https://launchpad.net/~webupd8team/+archive/ubuntu/java Press [ENTER] to continue or ctrl-c to cancel adding it gpg: keyring `/tmp/tmpFd_z7_/secring.gpg' created gpg: keyring `/tmp/tmpFd_z7_/pubring.gpg' created gpg: requesting key EEA14886 from hkp server keyserver.ubuntu.com gpg: /tmp/tmpFd_z7_/trustdb.gpg: trustdb created gpg: key EEA14886: public key "Launchpad VLC" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) OK user1@precise64:~$ user1@precise64:~$ sudo apt-get install oracle-java7-installer Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: binutils gsfonts gsfonts-x11 java-common libfontenc1 libfreetype6 libxfont1 wget x11-common xfonts-encodings xfonts-utils Suggested packages: binutils-doc default-jre equivs binfmt-support visualvm ttf-baekmuk ttf-unfonts ttf-unfonts-core ttf-kochi-gothic ttf-sazanami-gothic ttf-kochi-mincho ttf-sazanami-mincho ttf-arphic-uming firefox firefox-2 iceweasel mozilla-firefox iceape-browser mozilla-browser epiphany-gecko epiphany-webkit epiphany-browser galeon midbrowser moblin-web-browser xulrunner xulrunner-1.9 konqueror chromium-browser midori google-chrome The following NEW packages will be installed: binutils gsfonts gsfonts-x11 java-common libfontenc1 libfreetype6 libxfont1 oracle-java7-installer wget x11-common xfonts-encodings xfonts-utils 0 upgraded, 12 newly installed, 0 to remove and 0 not upgraded. Need to get 7,609 kB of archives. After this operation, 17.8 MB of additional disk space will be used. Do you want to continue [Y/n]? \\ セットアップされました。 user1@precise64:~$ java -version java version "1.7.0_65" Java(TM) SE Runtime Environment (build 1.7.0_65-b17) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) user1@precise64:~$ \\ ==== elasticsearch のセットアップ ==== [[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html]] の説明のとおり、セットアップします。 user1@precise64:~$ wget -qO - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add - OK user1@precise64:~$ \\ リポジトリを追加します。 user1@precise64:~$ sudo nano -w /etc/apt/sources.list deb http://archive.ubuntu.com/ubuntu precise main restricted universe multiverse deb http://archive.ubuntu.com/ubuntu precise-updates main restricted universe multiverse deb http://security.ubuntu.com/ubuntu precise-security main restricted universe multiverse deb http://packages.elasticsearch.org/elasticsearch/1.3/debian stable main \\ elasticsearch をインストールします。 user1@precise64:~$ sudo apt-get update user1@precise64:~$ sudo apt-get install elasticsearch Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: elasticsearch 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 26.9 MB of archives. After this operation, 30.9 MB of additional disk space will be used. Get:1 http://packages.elasticsearch.org/elasticsearch/1.3/debian/ stable/main elasticsearch all 1.3.1 [26.9 MB] Fetched 26.9 MB in 5s (4,866 kB/s) Selecting previously unselected package elasticsearch. (Reading database ... 20533 files and directories currently installed.) Unpacking elasticsearch (from .../elasticsearch_1.3.1_all.deb) ... Processing triggers for ureadahead ... Setting up elasticsearch (1.3.1) ... Adding system user `elasticsearch' (UID 105) ... Adding new user `elasticsearch' (UID 105) with group `elasticsearch' ... Not creating home directory `/usr/share/elasticsearch'. ### NOT starting elasticsearch by default on bootup, please execute sudo update-rc.d elasticsearch defaults 95 10 ### In order to start elasticsearch, execute sudo /etc/init.d/elasticsearch start user1@precise64:~$ \\ ==== fluentd-ui のセットアップ ==== \\