Eucalyptus

Merge lp:~syuu1228/eucalyptus/rados4eucalyptus-devel into lp:eucalyptus

  1. rados4eucalyptus-devel
  2. Merge into eucalyptus-devel
Proposed by Takuya ASADA
Status: Needs review
Proposed branch: lp:~syuu1228/eucalyptus/rados4eucalyptus-devel
Merge into: lp:eucalyptus
Diff against target: 16619 lines (has conflicts) 
Text conflict in clc/build.xml
Contents conflict in clc/modules/database/conf/scripts/caches.groovy
To merge this branch: bzr merge lp:~syuu1228/eucalyptus/rados4eucalyptus-devel
Reviewer Review Type Date Requested Status
Neil Soman Pending
Review via email: mp+40838@code.launchpad.net

Description of the change

Hi,

I'm trying to integrate a distributed storage system named "RADOS" for both S3 service and EBS service on Eucalyptus, to make it scalable.

RADOS is a part of Ceph filesystem, which provides distributed object store.
It provides API for C/C++ and S3 compatible server, EBS compatible block device for qemu/kvm.
Details are described on this page:
http://ceph.newdream.net/2009/05/the-rados-distributed-object-store/

This patch is minimum implement of the work, it's implements SystemStorageManager, FileIO, ChunkedDataFile for RADOS.
This provides RADOS based Walrus backend, when it's enabled Walrus stores objects to RADOS cluster instead of local file system.

Existing local file system module is moved from storage-common to storage-fs, and new rados module added as storage-rados.
It can switch via configure option --with-rados=[rados home], if it specified storage-rados will install. Otherwise storage-fs will install.

And this patch doesn't include following features, it going to be another patches:
- Zerocopy on JNI
  Currently the implementation copy buffer between librados and Java code, we need to prevent it to get better performance.
  To make it zerocopy, we need to modify Ceph implementation, not only my code.
  I'm discussing it on Ceph ML now, but it takes few more time.
- Multiple Walrus support for CLC
  We need to support multiple Walrus to make it scalable.
  I just implemented "quick hack" support Multiple Walrus, but I need to make it better before posting the patch.
- RADOS based EBS(rbd) support
  I only worked on Walrus now, but I also would like to work on rbd support for Storage Controller.

*Performance test1: chunk size
On storage-rados, default chunk size(8KB) in StorageManager.sendObject() is too small.
Here's throughput on default chunk size:
- storage-fs: 33.34MB/s
- storage-rados: 4.81MB/s

And here's read throughput when extending chunk size from 8KB to 80MB on storage-rados:
8K 4.81MB/s
80K 13.37MB/s
800K 29.52MB/s
1M 31.58MB/s
2M 33.54MB/s
3M 35.19MB/s
4M 35.67MB/s
5M 37.49MB/s
6M 35MB/s
8M 33.65MB/s
80M 23.85MB/s

So we need to change the default chunk size to 5MB on storage-rados.
I also measured 5MB on rados-fs for comparison, it's 65.60MB/s.
That means changing the default chunk size also makes rados-fs faster anyway.

Testing environment as follows:
Throughput are measured by s3cmd from Gigabit ethernet, same segment with Walrus.
RADOS cluster constructed with 9 nodes, 1 node for monitor, the others are storage(OSD).
Test file size is 1GB, single file.

[node assignment]
node0: CC/CLC, s3cmd
node1: Walrus, RADOS Monitor
node2: RADOS Storage
node3: RADOS Storage
node4: RADOS Storage
node5: RADOS Storage
node6: RADOS Storage
node7: RADOS Storage
node8: RADOS Storage
node9: RADOS Storage

[node spec]
CPU: Athlon II X4 605e
Memory: 16GB
HDD: SATA 250GB via Areca SATA Host Adapter RAID Controller
OS: Ubuntu Server 10.04

*Performance test2: scalability
I also measured the throughput when requesting multiple read request concurrently.
(This actually requires multiple Walrus implementation which doesn't include the patch, as I described earlier)

Test condition as follows:
Compared the performance when number of Walrus node is 1, 2, 4, 8 using storage-rados, and also storage-fs.
Storage node for RADOS is always 8 node, sharing Walrus node.
Read requests are sending from 2 nodes, each node sends 1, 2, 4, 8, 16, 32, 64 requests concurrently.
So overall requests are 2, 4, 8, 16, 32, 64, 128.

The graph is on following URL:
http://cid-35288454e2692e6b.photos.live.com/self.aspx/public/graph.png
Y-axis is average throughput(per request), unit is MB/s.
X-axis is number of request.

From the graph we can see the system scaling when adding Walrus nodes.
Even on 1 Walrus node, it's faster than storage-fs. This probably means 8 nodes of storage cluster is faster than local filesystem when multiple requests occurred.

Testing environment as follows:
Throughput are measured by modified s3cmd from Gigabit ethernet, same segment with Walrus, 2 nodes.
RADOS cluster constructed 1 node for monitor, 8 nodes for storage(OSD).
Test file size is 10MB, 64 files.

[node assignment]
node0: CC/CLC, s3cmd
node1: Walrus, RADOS Monitor, RADOS Storage
node2: Walrus, RADOS Storage
node3: Walrus, RADOS Storage
node4: Walrus, RADOS Storage
node5: Walrus, RADOS Storage
node6: Walrus, RADOS Storage
node7: Walrus, RADOS Storage
node8: Walrus, RADOS Storage
node9: s3cmd

[node spec]
same as test1

*Documents
Here's a document for install procedure:
http://r4eucalyptus.wikia.com/wiki/Installing_RADOS4Eucalyptus_2.0

To post a comment you must log in.

Unmerged revisions

1239. By tasada <tasada@7ae02a3f>

storage-rados module implemented. local filesystem classes moved to storage-rados module.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'Makedefs.in'
--- Makedefs.in 2010-10-22 17:50:41 +0000
+++ Makedefs.in 2010-11-15 08:38:14 +0000
@@ -11,6 +11,8 @@
11export AXIS2C_SERVICES = @AXIS2C_SERVICES@11export AXIS2C_SERVICES = @AXIS2C_SERVICES@
12export AXIS2_HOME = @AXIS2_HOME@12export AXIS2_HOME = @AXIS2_HOME@
13export LIBVIRT_HOME = @LIBVIRT_HOME@13export LIBVIRT_HOME = @LIBVIRT_HOME@
14export RADOS_HOME = @RADOS_HOME@
15export STORAGE_MODULE = @STORAGE_MODULE@
1416
15# where eucalyptus will install and other standard stuff: eventually we'll17# where eucalyptus will install and other standard stuff: eventually we'll
16# comply to the GNU standard18# comply to the GNU standard
1719
=== modified file 'clc/Makefile'
--- clc/Makefile 2010-06-02 20:29:20 +0000
+++ clc/Makefile 2010-11-15 08:38:14 +0000
@@ -50,27 +50,42 @@
50# echo "Cannot find java deps, please run 'make deps' in the 'clc/' subdirectory."; exit 1; fi50# echo "Cannot find java deps, please run 'make deps' in the 'clc/' subdirectory."; exit 1; fi
51 @ANT_OPTS="-Xmx512m" $(ANT) build51 @ANT_OPTS="-Xmx512m" $(ANT) build
52 @make -C modules/storage-controller/native52 @make -C modules/storage-controller/native
53 @if test "${STORAGE_MODULE}" = "storage-rados"; then \
54 make -C modules/storage-rados/native RADOS_HOME="${RADOS_HOME}"; \
55 fi
53 @make -C modules/bootstrap56 @make -C modules/bootstrap
5457
55clean:58clean:
56 $(ANT) clean59 $(ANT) clean
57 make -C modules/bootstrap clean60 make -C modules/bootstrap clean
61 @if test "${STORAGE_MODULE}" = "storage-rados"; then \
62 make -C modules/storage-rados/native clean; \
63 fi
58 make -C modules/storage-controller/native clean64 make -C modules/storage-controller/native clean
5965
60distclean: clean66distclean: clean
61 $(ANT) distclean67 $(ANT) distclean
62 make -C modules/bootstrap distclean68 make -C modules/bootstrap distclean
69 @if test "${STORAGE_MODULE}" = "storage-rados"; then \
70 make -C modules/storage-rados/native distclean; \
71 fi
63 make -C modules/storage-controller/native distclean72 make -C modules/storage-controller/native distclean
6473
65install: build74install: build
66 @$(ANT) install75 @$(ANT) install
67 @make -C modules/storage-controller/native install76 @make -C modules/storage-controller/native install
77 @if test "${STORAGE_MODULE}" = "storage-rados"; then \
78 make -C modules/storage-rados/native install; \
79 fi
68 @make -C modules/bootstrap install80 @make -C modules/bootstrap install
6981
70deploy: install82deploy: install
7183
72uninstall:84uninstall:
73 make -C modules/storage-controller/native uninstall85 make -C modules/storage-controller/native uninstall
86 @if test "${STORAGE_MODULE}" = "storage-rados"; then \
87 make -C modules/storage-rados/native uninstall; \
88 fi
74 make -C modules/bootstrap uninstall89 make -C modules/bootstrap uninstall
75 $(RM) -rf $(DESTDIR)$(etcdir)/eucalyptus/cloud.d90 $(RM) -rf $(DESTDIR)$(etcdir)/eucalyptus/cloud.d
76 $(RM) -rf $(DESTDIR)$(datarootdir)/eucalyptus/*jar91 $(RM) -rf $(DESTDIR)$(datarootdir)/eucalyptus/*jar
7792
=== modified file 'clc/build.xml'
--- clc/build.xml 2010-11-09 19:31:25 +0000
+++ clc/build.xml 2010-11-15 08:38:14 +0000
@@ -98,6 +98,7 @@
98 <ant dir="modules/cloud" inheritall="false" target="builder" />98 <ant dir="modules/cloud" inheritall="false" target="builder" />
99 <ant dir="modules/cluster-manager" inheritall="false" target="builder" />99 <ant dir="modules/cluster-manager" inheritall="false" target="builder" />
100 <ant dir="modules/storage-common" inheritall="false" target="builder" />100 <ant dir="modules/storage-common" inheritall="false" target="builder" />
101 <ant dir="modules/storage-rados" inheritall="false" target="builder" />
101 <ant dir="modules/walrus" inheritall="false" target="builder" />102 <ant dir="modules/walrus" inheritall="false" target="builder" />
102 <ant dir="modules/storage-controller" inheritall="false" target="builder" />103 <ant dir="modules/storage-controller" inheritall="false" target="builder" />
103 <ant dir="modules/dns" inheritall="false" target="builder" />104 <ant dir="modules/dns" inheritall="false" target="builder" />
@@ -122,6 +123,7 @@
122 <ant dir="modules/wsstack" inheritall="false" target="jar" />123 <ant dir="modules/wsstack" inheritall="false" target="jar" />
123 <ant dir="modules/cluster-manager" inheritall="false" target="jar" />124 <ant dir="modules/cluster-manager" inheritall="false" target="jar" />
124 <ant dir="modules/storage-common" inheritall="false" target="jar" />125 <ant dir="modules/storage-common" inheritall="false" target="jar" />
126 <ant dir="modules/storage-rados" inheritall="false" target="jar" />
125 <ant dir="modules/walrus" inheritall="false" target="jar" />127 <ant dir="modules/walrus" inheritall="false" target="jar" />
126 <ant dir="modules/storage-controller" inheritall="false" target="jar" />128 <ant dir="modules/storage-controller" inheritall="false" target="jar" />
127 <ant dir="modules/dns" inheritall="false" target="jar" />129 <ant dir="modules/dns" inheritall="false" target="jar" />
@@ -177,6 +179,7 @@
177 <ant dir="modules/wsstack" inheritall="false" target="clean" />179 <ant dir="modules/wsstack" inheritall="false" target="clean" />
178 <ant dir="modules/cluster-manager" inheritall="false" target="clean" />180 <ant dir="modules/cluster-manager" inheritall="false" target="clean" />
179 <ant dir="modules/storage-common" inheritall="false" target="clean" />181 <ant dir="modules/storage-common" inheritall="false" target="clean" />
182 <ant dir="modules/storage-rados" inheritall="false" target="clean" />
180 <ant dir="modules/walrus" inheritall="false" target="clean" />183 <ant dir="modules/walrus" inheritall="false" target="clean" />
181 <ant dir="modules/storage-controller" inheritall="false" target="clean" />184 <ant dir="modules/storage-controller" inheritall="false" target="clean" />
182 <ant dir="modules/dns" inheritall="false" target="clean" />185 <ant dir="modules/dns" inheritall="false" target="clean" />
@@ -244,6 +247,11 @@
244 <ant dir="modules/core" inheritall="false" target="install" />247 <ant dir="modules/core" inheritall="false" target="install" />
245 <ant dir="modules/wsstack" inheritall="false" target="install" />248 <ant dir="modules/wsstack" inheritall="false" target="install" />
246 <ant dir="modules/storage-common" inheritall="false" target="install" />249 <ant dir="modules/storage-common" inheritall="false" target="install" />
250<<<<<<< TREE
251=======
252 <ant dir="modules/storage-rados" inheritall="false" target="install" />
253 <ant dir="modules/interface" inheritall="false" target="install" />
254>>>>>>> MERGE-SOURCE
247 <ant dir="modules/cluster-manager" inheritall="false" target="install" />255 <ant dir="modules/cluster-manager" inheritall="false" target="install" />
248 <ant dir="modules/walrus" inheritall="false" target="install" />256 <ant dir="modules/walrus" inheritall="false" target="install" />
249 <ant dir="modules/storage-controller" inheritall="false" target="install" />257 <ant dir="modules/storage-controller" inheritall="false" target="install" />
250258
=== added file 'clc/build.xml.in'
--- clc/build.xml.in 1970-01-01 00:00:00 +0000
+++ clc/build.xml.in 2010-11-15 08:38:14 +0000
@@ -0,0 +1,292 @@
1<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 ~ Copyright (c) 2009 Eucalyptus Systems, Inc.
3 ~
4 ~ This program is free software: you can redistribute it and/or modify
5 ~ it under the terms of the GNU General Public License as published by
6 ~ the Free Software Foundation, only version 3 of the License.
7 ~
8 ~
9 ~ This file is distributed in the hope that it will be useful, but WITHOUT
10 ~ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11 ~ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12 ~ for more details.
13 ~
14 ~ You should have received a copy of the GNU General Public License along
15 ~ with this program. If not, see http://www.gnu.org/licenses/.
16 ~
17 ~ Please contact Eucalyptus Systems, Inc., 130 Castilian
18 ~ Dr., Goleta, CA 93101 USA or visit http://www.eucalyptus.com/licenses/
19 ~ if you need additional information or have any questions.
20 ~
21 ~ This file may incorporate work covered under the following copyright and
22 ~ permission notice:
23 ~
24 ~ Software License Agreement (BSD License)
25 ~
26 ~ Copyright (c) 2008, Regents of the University of California
27 ~ All rights reserved.
28 ~
29 ~ Redistribution and use of this software in source and binary forms, with
30 ~ or without modification, are permitted provided that the following
31 ~ conditions are met:
32 ~
33 ~ Redistributions of source code must retain the above copyright notice,
34 ~ this list of conditions and the following disclaimer.
35 ~
36 ~ Redistributions in binary form must reproduce the above copyright
37 ~ notice, this list of conditions and the following disclaimer in the
38 ~ documentation and/or other materials provided with the distribution.
39 ~
40 ~ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41 ~ IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42 ~ TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43 ~ PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44 ~ OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45 ~ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46 ~ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47 ~ PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48 ~ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49 ~ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50 ~ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51 ~ THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52 ~ LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53 ~ SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54 ~ IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55 ~ BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56 ~ THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57 ~ OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58 ~ WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59 ~ ANY SUCH LICENSES OR RIGHTS.
60 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
61 ~ Author: chris grzegorczyk grze@eucalyptus.com
62 -->
63
64<project name="eucalyptus" basedir=".">
65 <property environment="localenv" />
66 <property name="euca.home.0" value="${localenv.EUCALYPTUS}" />
67 <condition property="euca.home" value="/opt/eucalyptus" else="${localenv.EUCALYPTUS}">
68 <contains string="${euca.home.0}" substring="localenv" />
69 </condition>
70 <property name="DESTDIR.0" value="${localenv.DESTDIR}" />
71 <condition property="DESTDIR" value="/" else="${localenv.DESTDIR}">
72 <contains string="${DESTDIR.0}" substring="localenv" />
73 </condition>
74 <property name="project.basedir" value="${user.dir}" />
75 <loadfile srcfile="${project.basedir}/../VERSION" property="euca.version">
76 <filterchain>
77 <filterreader classname="org.apache.tools.ant.filters.StripLineBreaks" />
78 </filterchain>
79 </loadfile>
80 <!--================================== other vars ==================================-->
81 <property name="jvm.heap" value="256m" />
82 <property name="deps.url" value="http://eucalyptussoftware.com/downloads/dependencies/${euca.version}" />
83 <property name="deps.dir" value="deps" />
84 <property name="deps.lib.dir" value="lib" />
85 <property name="tools.dir" value="tools/src" />
86 <property name="deps.lib" value="cloud-lib" />
87 <property name="bind.timestamp" value="target/.bind-timestamp" />
88 <!--================================== build targets ==================================-->
89 <target name="build">
90 <ant dir="modules/msgs" inheritall="false" target="builder" />
91 <ant dir="modules/database" inheritall="false" target="builder" />
92 <ant dir="modules/hsqldb" inheritall="false" target="builder" />
93 <ant dir="modules/authentication" inheritall="false" target="builder" />
94 <ant dir="modules/cloud" inheritall="false" target="builder" />
95 <ant dir="modules/configuration" inheritall="false" target="builder" />
96 <ant dir="modules/core" inheritall="false" target="builder" />
97 <ant dir="modules/wsstack" inheritall="false" target="builder" />
98 <ant dir="modules/component" inheritall="false" target="builder" />
99 <ant dir="modules/interface" inheritall="false" target="builder" />
100 <ant dir="modules/group-manager" inheritall="false" target="builder" />
101 <ant dir="modules/cluster-manager" inheritall="false" target="builder" />
102 <ant dir="modules/image-manager" inheritall="false" target="builder" />
103 <ant dir="modules/key-manager" inheritall="false" target="builder" />
104 <ant dir="modules/storage-common" inheritall="false" target="builder" />
105 <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="builder" />
106 <ant dir="modules/walrus" inheritall="false" target="builder" />
107 <ant dir="modules/storage-controller" inheritall="false" target="builder" />
108 <ant dir="modules/dns" inheritall="false" target="builder" />
109 <ant dir="modules/www" inheritall="false" target="builder" />
110 <ant dir="." inheritall="false" target="jar" />
111 </target>
112 <target name="jar" depends="bind">
113 <ant dir="modules/msgs" inheritall="false" target="jar" />
114 <ant dir="modules/database" inheritall="false" target="jar" />
115 <ant dir="modules/hsqldb" inheritall="false" target="jar" />
116 <ant dir="modules/authentication" inheritall="false" target="jar" />
117 <ant dir="modules/cloud" inheritall="false" target="jar" />
118 <ant dir="modules/configuration" inheritall="false" target="jar" />
119 <ant dir="modules/component" inheritall="false" target="jar" />
120 <ant dir="modules/core" inheritall="false" target="jar" />
121 <ant dir="modules/wsstack" inheritall="false" target="jar" />
122 <ant dir="modules/interface" inheritall="false" target="jar" />
123 <ant dir="modules/group-manager" inheritall="false" target="jar" />
124 <ant dir="modules/cluster-manager" inheritall="false" target="jar" />
125 <ant dir="modules/image-manager" inheritall="false" target="jar" />
126 <ant dir="modules/key-manager" inheritall="false" target="jar" />
127 <ant dir="modules/storage-common" inheritall="false" target="jar" />
128 <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="jar" />
129 <ant dir="modules/walrus" inheritall="false" target="jar" />
130 <ant dir="modules/storage-controller" inheritall="false" target="jar" />
131 <ant dir="modules/dns" inheritall="false" target="jar" />
132 <ant dir="modules/www" inheritall="false" target="jar" />
133 </target>
134 <target name="shouldBind">
135 <uptodate property="bind.notRequired" targetfile="${bind.timestamp}">
136 <srcfiles dir="modules">
137 <include name="**/*.java"/>
138 </srcfiles>
139 </uptodate>
140 <echo message="[BINDING] Up-to-date check shows bind.notRequred=${bind.notRequired}." />
141 </target>
142 <target name="bind" depends="shouldBind" unless="bind.notRequired">
143 <echo message="[BINDING] Running binding compiler." />
144 <path id="bindingclasspath">
145 <dirset dir="modules">
146 <include name="**/build" />
147 </dirset>
148 <fileset dir="lib">
149 <include name="**/*.jar" />
150 </fileset>
151 </path>
152 <delete>
153 <fileset dir="modules" includes="**/*JiBX*" />
154 </delete>
155 <delete file="modules/msgs/src/main/resources/msgs-binding.xml" />
156 <taskdef name="bindings" classname="com.eucalyptus.binding.BuildBindings" classpathref="bindingclasspath" />
157 <bindings>
158 <classfileset dir="${project.basedir}/modules" includes="*/build/**" />
159 <bindingfileset dir="${project.basedir}/modules" includes="**/*-binding.xml" />
160 </bindings>
161 <touch file="${bind.timestamp}"/>
162 </target>
163
164 <!--================================== clean targets ==================================-->
165 <target name="clean">
166 <delete file="${bind.timestamp}"/>
167 <ant dir="modules/msgs" inheritall="false" target="clean" />
168 <ant dir="modules/database" inheritall="false" target="clean" />
169 <ant dir="modules/hsqldb" inheritall="false" target="clean" />
170 <ant dir="modules/cloud" inheritall="false" target="clean" />
171 <ant dir="modules/authentication" inheritall="false" target="clean" />
172 <ant dir="modules/configuration" inheritall="false" target="clean" />
173 <ant dir="modules/component" inheritall="false" target="clean" />
174 <ant dir="modules/core" inheritall="false" target="clean" />
175 <ant dir="modules/wsstack" inheritall="false" target="clean" />
176 <ant dir="modules/interface" inheritall="false" target="clean" />
177 <ant dir="modules/cluster-manager" inheritall="false" target="clean" />
178 <ant dir="modules/image-manager" inheritall="false" target="clean" />
179 <ant dir="modules/group-manager" inheritall="false" target="clean" />
180 <ant dir="modules/key-manager" inheritall="false" target="clean" />
181 <ant dir="modules/storage-common" inheritall="false" target="clean" />
182 <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="clean" />
183 <ant dir="modules/walrus" inheritall="false" target="clean" />
184 <ant dir="modules/storage-controller" inheritall="false" target="clean" />
185 <ant dir="modules/dns" inheritall="false" target="clean" />
186 <ant dir="modules/www" inheritall="false" target="clean" />
187 <delete dir="${target.dir}" />
188 </target>
189 <target name="distclean" depends="clean">
190 <delete dir="${deps.dir}" />
191 <delete dir="${deps.lib.dir}" />
192 <delete dir="target" />
193 </target>
194 <!--================================== install target ==================================-->
195 <property name="euca.conf.dir" value="${euca.home}/etc/eucalyptus/cloud.d" />
196 <property name="euca.lib.dir" value="${euca.home}/usr/share/eucalyptus" />
197 <property name="euca.var.dir" value="${euca.home}/var/lib/eucalyptus" />
198 <property name="euca.run.dir" value="${euca.home}/var/run/eucalyptus" />
199 <property name="euca.log.dir" value="${euca.home}/var/log/eucalyptus" />
200 <property name="euca.bin.dir" value="${euca.home}/usr/sbin" />
201 <target name="install">
202 <!-- copy libraries over -->
203 <mkdir dir="${DESTDIR}${euca.lib.dir}" />
204 <copy todir="${DESTDIR}${euca.lib.dir}" overwrite="true">
205 <fileset dir="${deps.lib.dir}">
206 <include name="**/*" />
207 <exclude name="**/gwt-api-checker-*.jar" />
208 <exclude name="**/gwt-dev-*.jar" />
209 <exclude name="**/gwt-soyc-vis-*.jar" />
210 <exclude name="**/gwt-user-*.jar" />
211 </fileset>
212 </copy>
213 <!--copy admin tools -->
214 <mkdir dir="${DESTDIR}${euca.bin.dir}" />
215 <copy todir="${DESTDIR}${euca.bin.dir}" overwrite="true">
216 <fileset dir="${tools.dir}">
217 <include name="**/*" />
218 </fileset>
219 </copy>
220 <chmod dir="${DESTDIR}${euca.bin.dir}" perm="0755">
221 <include name="**/euca-*" />
222 </chmod>
223 <mkdir dir="${DESTDIR}${euca.conf.dir}" />
224 <chmod dir="${DESTDIR}${euca.conf.dir}" perm="0755" />
225 <mkdir dir="${DESTDIR}${euca.var.dir}/" />
226 <mkdir dir="${DESTDIR}${euca.var.dir}/db" />
227 <mkdir dir="${DESTDIR}${euca.var.dir}/keys" />
228 <mkdir dir="${DESTDIR}${euca.log.dir}" />
229 <mkdir dir="${DESTDIR}${euca.var.dir}/webapps" />
230 <mkdir dir="${DESTDIR}${euca.var.dir}/modules" />
231 <mkdir dir="${DESTDIR}${euca.run.dir}" />
232 <ant dir="modules/msgs" inheritall="false" target="install" />
233 <ant dir="modules/database" inheritall="false" target="install" />
234 <ant dir="modules/hsqldb" inheritall="false" target="install" />
235 <ant dir="modules/cloud" inheritall="false" target="install" />
236 <ant dir="modules/authentication" inheritall="false" target="install" />
237 <ant dir="modules/configuration" inheritall="false" target="install" />
238 <ant dir="modules/component" inheritall="false" target="install" />
239 <ant dir="modules/core" inheritall="false" target="install" />
240 <ant dir="modules/wsstack" inheritall="false" target="install" />
241 <ant dir="modules/storage-common" inheritall="false" target="install" />
242 <ant dir="modules/@STORAGE_MODULE@" inheritall="false" target="install" />
243 <ant dir="modules/interface" inheritall="false" target="install" />
244 <ant dir="modules/cluster-manager" inheritall="false" target="install" />
245 <ant dir="modules/image-manager" inheritall="false" target="install" />
246 <ant dir="modules/group-manager" inheritall="false" target="install" />
247 <ant dir="modules/key-manager" inheritall="false" target="install" />
248 <ant dir="modules/walrus" inheritall="false" target="install" />
249 <ant dir="modules/storage-controller" inheritall="false" target="install" />
250 <ant dir="modules/dns" inheritall="false" target="install" />
251 <ant dir="modules/www" inheritall="false" target="install" />
252 </target>
253 <!--================================== unzipping target ==================================-->
254 <target name="deps">
255 <condition property="needDownload">
256 <and>
257 <not>
258 <available file="${deps.lib.dir}/bcprov.jar" />
259 </not>
260 <and>
261 <not>
262 <available file="${deps.dir}/${deps.lib}.tar.gz" />
263 </not>
264 </and>
265 </and>
266 </condition>
267 <ant target="download-deps" />
268 <condition property="needUnzipping">
269 <and>
270 <not>
271 <available file="${deps.lib.dir}/bcprov.jar" />
272 </not>
273 <and>
274 <available file="${deps.dir}/${deps.lib}.tar.gz" />
275 </and>
276 </and>
277 </condition>
278 <ant target="untar" />
279 </target>
280 <target name="dist-deps" depends="deps">
281 <delete dir="${deps.dir}" />
282 </target>
283 <target name="untar" if="needUnzipping">
284 <untar compression="gzip" src="${deps.dir}/${deps.lib}.tar.gz" dest="${deps.lib.dir}" />
285 </target>
286 <target name="download-deps" if="needDownload">
287 <mkdir dir="${deps.dir}" />
288 <mkdir dir="${deps.lib.dir}" />
289 <!--=========== download the dependency libraries ===========-->
290 <get src="${deps.url}/${deps.lib}.tar.gz" dest="${deps.dir}/${deps.lib}.tar.gz" />
291 </target>
292</project>
0293
=== added file 'clc/modules/database/conf/scripts/caches.groovy.OTHER'
--- clc/modules/database/conf/scripts/caches.groovy.OTHER 1970-01-01 00:00:00 +0000
+++ clc/modules/database/conf/scripts/caches.groovy.OTHER 2010-11-15 08:38:14 +0000
@@ -0,0 +1,7 @@
1cacheProps = [
2 'hibernate.cache.provider_class': 'net.sf.ehcache.hibernate.SingletonEhCacheProvider',
3 'hibernate.cache.region_prefix': "eucalyptus_${context_name}_cache",
4 'hibernate.cache.use_second_level_cache': 'false',
5 'hibernate.cache.use_query_cache': 'false',
6 'hibernate.cache.use_structured_entries': 'false',
7]
08
=== removed directory 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud'
=== removed directory 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws'
=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 2009-08-27 18:38:18 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 1970-01-01 00:00:00 +0000
@@ -1,77 +0,0 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package edu.ucsb.eucalyptus.cloud.ws;
62
63import java.io.IOException;
64import java.io.RandomAccessFile;
65
66import org.jboss.netty.handler.stream.ChunkedFile;
67
68import edu.ucsb.eucalyptus.constants.IsData;
69
70public class ChunkedDataFile extends ChunkedFile implements IsData {
71
72 public ChunkedDataFile(RandomAccessFile file, long offset, long length,
73 int chunkSize) throws IOException {
74 super(file, offset, length, chunkSize);
75 }
76}
77
780
=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 2009-08-27 18:38:18 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 1970-01-01 00:00:00 +0000
@@ -1,120 +0,0 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package edu.ucsb.eucalyptus.cloud.ws;
62
63import java.io.ByteArrayOutputStream;
64import java.io.RandomAccessFile;
65import java.util.zip.GZIPOutputStream;
66
67import org.apache.log4j.Logger;
68import org.jboss.netty.buffer.ChannelBuffers;
69import org.jboss.netty.handler.stream.ChunkedInput;
70
71import edu.ucsb.eucalyptus.constants.IsData;
72
73public class CompressedChunkedFile implements ChunkedInput, IsData {
74 private Logger LOG = Logger.getLogger( CompressedChunkedFile.class );
75 private RandomAccessFile file;
76 private long offset;
77 private int CHUNK_SIZE = 8192;
78 private long fileLength;
79
80 public CompressedChunkedFile(RandomAccessFile file, long fileLength) {
81 this.file = file;
82 this.offset = 0;
83 this.fileLength = fileLength;
84 }
85
86 public CompressedChunkedFile(RandomAccessFile file, long start, long end, int chunkSize) {
87 this.file = file;
88 this.offset = start;
89 this.fileLength = end;
90 CHUNK_SIZE = chunkSize;
91 }
92
93 @Override
94 public void close() throws Exception {
95 file.close();
96 }
97
98 @Override
99 public boolean hasNextChunk() throws Exception {
100 return offset < fileLength && file.getChannel().isOpen();
101 }
102
103 @Override
104 public Object nextChunk() throws Exception {
105 long offset = this.offset;
106 if (offset >= fileLength) {
107 return null;
108 }
109 int chunkSize = (int) Math.min(CHUNK_SIZE, fileLength - offset);
110 byte[] chunk = new byte[chunkSize];
111 file.readFully(chunk);
112 this.offset = offset + chunkSize;
113 ByteArrayOutputStream out = new ByteArrayOutputStream();
114 GZIPOutputStream zip = new GZIPOutputStream(out);
115 zip.write(chunk);
116 zip.close();
117 return ChannelBuffers.wrappedBuffer(out.toByteArray());
118 }
119}
120
1210
=== added file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageIO.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageIO.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageIO.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,80 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage;
67
68import java.io.IOException;
69import java.nio.ByteBuffer;
70
71public abstract class StorageIO {
72
73 public abstract int read(long offset) throws IOException;
74
75 public abstract void write(byte[] bytes) throws IOException;
76
77 public abstract ByteBuffer getBuffer();
78
79 public abstract void finish();
80}
081
=== modified file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageManager.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageManager.java 2010-04-26 20:50:11 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/StorageManager.java 2010-11-15 08:38:14 +0000
@@ -75,7 +75,6 @@
7575
76import edu.ucsb.eucalyptus.cloud.BucketLogData;76import edu.ucsb.eucalyptus.cloud.BucketLogData;
77import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;77import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
78import edu.ucsb.eucalyptus.storage.fs.FileIO;
7978
80public interface StorageManager {79public interface StorageManager {
8180
@@ -95,9 +94,9 @@
9594
96 public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException;95 public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException;
9796
98 public FileIO prepareForRead(String bucket, String object) throws Exception;97 public StorageIO prepareForRead(String bucket, String object) throws Exception;
9998
100 public FileIO prepareForWrite(String bucket, String object) throws Exception;99 public StorageIO prepareForWrite(String bucket, String object) throws Exception;
101100
102 public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException;101 public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException;
103102
104103
=== removed directory 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs'
=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileIO.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileIO.java 2009-08-27 18:38:18 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileIO.java 1970-01-01 00:00:00 +0000
@@ -1,83 +0,0 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage.fs;
67
68import java.io.IOException;
69import java.nio.ByteBuffer;
70import java.nio.channels.FileChannel;
71
72public abstract class FileIO {
73
74 protected FileChannel channel;
75
76 public abstract int read(long offset) throws IOException;
77
78 public abstract void write(byte[] bytes) throws IOException;
79
80 public abstract ByteBuffer getBuffer();
81
82 public abstract void finish();
83}
840
=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 2009-10-02 17:47:47 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 1970-01-01 00:00:00 +0000
@@ -1,110 +0,0 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage.fs;
67
68import org.apache.log4j.Logger;
69
70import java.io.FileInputStream;
71import java.io.FileNotFoundException;
72import java.io.IOException;
73import java.nio.ByteBuffer;
74
75public class FileReader extends FileIO {
76
77 private static Logger LOG = Logger.getLogger(FileReader.class);
78 private ByteBuffer buffer;
79
80 public FileReader(String filename) throws Exception {
81 try {
82 channel = new FileInputStream(filename).getChannel();
83 buffer = ByteBuffer.allocate(102400/*TODO: NEIL WalrusQueryDispatcher.DATA_MESSAGE_SIZE */);
84 } catch( FileNotFoundException ex) {
85 LOG.error(ex);
86 throw ex;
87 }
88 }
89
90 public int read(long offset) throws IOException {
91 buffer.clear();
92 int bytesRead = channel.read(buffer, offset);
93 buffer.flip();
94 return bytesRead;
95 }
96
97 public void write(byte[] bytes) throws IOException {}
98
99 public ByteBuffer getBuffer() {
100 return buffer;
101 }
102
103 public void finish() {
104 try {
105 channel.close();
106 } catch(IOException ex) {
107 LOG.error(ex);
108 }
109 }
110}
1110
=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 2010-11-03 19:16:45 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 1970-01-01 00:00:00 +0000
@@ -1,465 +0,0 @@
1/*******************************************************************************
2 *Copyright (c) 2009 Eucalyptus Systems, Inc.
3 *
4 * This program is free software: you can redistribute it and/or modify
5 * it under the terms of the GNU General Public License as published by
6 * the Free Software Foundation, only version 3 of the License.
7 *
8 *
9 * This file is distributed in the hope that it will be useful, but WITHOUT
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12 * for more details.
13 *
14 * You should have received a copy of the GNU General Public License along
15 * with this program. If not, see <http://www.gnu.org/licenses/>.
16 *
17 * Please contact Eucalyptus Systems, Inc., 130 Castilian
18 * Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19 * if you need additional information or have any questions.
20 *
21 * This file may incorporate work covered under the following copyright and
22 * permission notice:
23 *
24 * Software License Agreement (BSD License)
25 *
26 * Copyright (c) 2008, Regents of the University of California
27 * All rights reserved.
28 *
29 * Redistribution and use of this software in source and binary forms, with
30 * or without modification, are permitted provided that the following
31 * conditions are met:
32 *
33 * Redistributions of source code must retain the above copyright notice,
34 * this list of conditions and the following disclaimer.
35 *
36 * Redistributions in binary form must reproduce the above copyright
37 * notice, this list of conditions and the following disclaimer in the
38 * documentation and/or other materials provided with the distribution.
39 *
40 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42 * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43 * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44 * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51 * THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52 * LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53 * SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54 * IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55 * BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56 * THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57 * OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58 * WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59 * ANY SUCH LICENSES OR RIGHTS.
60 *******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage.fs;
67
68import com.eucalyptus.auth.util.Hashes;
69import com.eucalyptus.context.Contexts;
70import com.eucalyptus.http.MappingHttpResponse;
71import com.eucalyptus.util.EucalyptusCloudException;
72import com.eucalyptus.util.ExecutionException;
73import com.eucalyptus.util.WalrusProperties;
74import com.eucalyptus.ws.util.ChannelUtil;
75import com.eucalyptus.ws.util.WalrusBucketLogger;
76
77import edu.ucsb.eucalyptus.cloud.BucketLogData;
78import edu.ucsb.eucalyptus.cloud.ws.CompressedChunkedFile;
79import edu.ucsb.eucalyptus.cloud.ws.ChunkedDataFile;
80import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
81import edu.ucsb.eucalyptus.storage.StorageManager;
82import edu.ucsb.eucalyptus.util.StreamConsumer;
83import edu.ucsb.eucalyptus.util.SystemUtil;
84
85import org.apache.log4j.Logger;
86import org.jboss.netty.channel.Channel;
87import org.jboss.netty.channel.ChannelFuture;
88import org.jboss.netty.channel.ChannelFutureListener;
89import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
90import org.jboss.netty.handler.codec.http.HttpHeaders;
91import org.jboss.netty.handler.stream.ChunkedInput;
92
93import java.io.*;
94import java.nio.channels.FileChannel;
95import java.util.ArrayList;
96import java.util.List;
97
98public class FileSystemStorageManager implements StorageManager {
99
100 public static final String FILE_SEPARATOR = "/";
101 public static final String lvmRootDirectory = "/dev";
102 private static boolean initialized = false;
103 private static String eucaHome = "/opt/eucalyptus";
104 public static final String EUCA_ROOT_WRAPPER = "/usr/lib/eucalyptus/euca_rootwrap";
105 public static final int MAX_LOOP_DEVICES = 256;
106 private static Logger LOG = Logger.getLogger(FileSystemStorageManager.class);
107
108 private String rootDirectory;
109 public FileSystemStorageManager(String rootDirectory) {
110 this.rootDirectory = rootDirectory;
111 }
112
113 public void checkPreconditions() throws EucalyptusCloudException {
114 try {
115 String eucaHomeDir = System.getProperty("euca.home");
116 if(eucaHomeDir == null) {
117 throw new EucalyptusCloudException("euca.home not set");
118 }
119 eucaHome = eucaHomeDir;
120 if(!new File(eucaHome + EUCA_ROOT_WRAPPER).exists()) {
121 throw new EucalyptusCloudException("root wrapper (euca_rootwrap) does not exist");
122 }
123 String returnValue = getLvmVersion();
124 if(returnValue.length() == 0) {
125 throw new EucalyptusCloudException("Is lvm installed?");
126 } else {
127 LOG.info(returnValue);
128 }
129 } catch(ExecutionException ex) {
130 String error = "Unable to run command: " + ex.getMessage();
131 LOG.error(error);
132 throw new EucalyptusCloudException(error);
133 }
134 }
135
136 public void setRootDirectory(String rootDirectory) {
137 this.rootDirectory = rootDirectory;
138 }
139
140 public boolean bucketExists(String bucket) {
141 return new File (rootDirectory + FILE_SEPARATOR + bucket).exists();
142 }
143
144 public boolean objectExists(String bucket, String object) {
145 return new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object).exists();
146 }
147
148 public void createBucket(String bucket) throws IOException {
149 File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
150 if(!bukkit.exists()) {
151 if(!bukkit.mkdirs()) {
152 throw new IOException("Unable to create bucket: " + bucket);
153 }
154 }
155 }
156
157 public long getSize(String bucket, String object) {
158 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
159 if(objectFile.exists())
160 return objectFile.length();
161 return -1;
162 }
163
164 public void deleteBucket(String bucket) throws IOException {
165 File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
166 if(!bukkit.delete()) {
167 throw new IOException("Unable to delete bucket: " + bucket);
168 }
169 }
170
171 public void createObject(String bucket, String object) throws IOException {
172 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
173 if (!objectFile.exists()) {
174 if (!objectFile.createNewFile()) {
175 throw new IOException("Unable to create: " + objectFile.getAbsolutePath());
176 }
177 }
178 }
179
180 public FileIO prepareForRead(String bucket, String object) throws Exception {
181 return new FileReader(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
182 }
183
184 public FileIO prepareForWrite(String bucket, String object) throws Exception {
185 return new FileWriter(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
186 }
187
188 public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException {
189 return readObject(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object, bytes, offset);
190 }
191
192 public int readObject(String path, byte[] bytes, long offset) throws IOException {
193 File objectFile = new File (path);
194 if (!objectFile.exists()) {
195 throw new IOException("Unable to read: " + path);
196 }
197 BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream(objectFile));
198 if (offset > 0) {
199 inputStream.skip(offset);
200 }
201 int bytesRead = inputStream.read(bytes);
202 inputStream.close();
203 return bytesRead;
204 }
205
206 public void deleteObject(String bucket, String object) throws IOException {
207 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
208 if (objectFile.exists()) {
209 if(!objectFile.delete()) {
210 throw new IOException("Unable to delete: " + objectFile.getAbsolutePath());
211 }
212 }
213 }
214
215 public void deleteAbsoluteObject(String object) throws IOException {
216 File objectFile = new File (object);
217 if (objectFile.exists()) {
218 if(!objectFile.delete()) {
219 throw new IOException("Unable to delete: " + object);
220 }
221 }
222 }
223
224 public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException {
225 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
226 if (!objectFile.exists()) {
227 objectFile.createNewFile();
228 }
229 BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(objectFile, append));
230 outputStream.write(base64Data);
231 outputStream.close();
232 }
233
234 public void renameObject(String bucket, String oldName, String newName) throws IOException {
235 File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + oldName);
236 File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + newName);
237 if(oldObjectFile.exists()) {
238 if (!oldObjectFile.renameTo(newObjectFile)) {
239 throw new IOException("Unable to rename " + oldObjectFile.getAbsolutePath() + " to " + newObjectFile.getAbsolutePath());
240 }
241 }
242 }
243
244 public void copyObject(String sourceBucket, String sourceObject, String destinationBucket, String destinationObject) throws IOException {
245 File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + sourceBucket + FILE_SEPARATOR + sourceObject);
246 File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + destinationBucket + FILE_SEPARATOR + destinationObject);
247 if(!oldObjectFile.equals(newObjectFile)) {
248 FileInputStream fileInputStream = new FileInputStream(oldObjectFile);
249 FileChannel fileIn = fileInputStream.getChannel();
250 FileOutputStream fileOutputStream = new FileOutputStream(newObjectFile);
251 FileChannel fileOut = fileOutputStream.getChannel();
252 fileIn.transferTo(0, fileIn.size(), fileOut);
253 fileIn.close();
254 fileInputStream.close();
255 fileOut.close();
256 fileOutputStream.close();
257 }
258 }
259
260 public String getObjectPath(String bucket, String object) {
261 return rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
262 }
263
264 public long getObjectSize(String bucket, String object) {
265 String absoluteObjectPath = rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
266
267 File objectFile = new File(absoluteObjectPath);
268 if(objectFile.exists())
269 return objectFile.length();
270 return -1;
271 }
272
273 public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
274 try {
275 Channel channel = request.getChannel();
276 RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
277 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
278 if(etag != null)
279 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
280 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
281 if(contentDisposition != null)
282 httpResponse.addHeader("Content-Disposition", contentDisposition);
283 final ChunkedInput file;
284 isCompressed = isCompressed == null ? false : isCompressed;
285 if(isCompressed) {
286 file = new CompressedChunkedFile(raf, size);
287 } else {
288 file = new ChunkedDataFile(raf, 0, size, 8192);
289 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
290 }
291 if(logData != null) {
292 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
293 logData.setBytesSent(size);
294 }
295 if(versionId != null) {
296 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
297 }
298 channel.write(httpResponse);
299 channel.write(file).addListener(new ChannelFutureListener( ) {
300 @Override public void operationComplete( ChannelFuture future ) throws Exception {
301 Contexts.clear(request.getCorrelationId());
302 file.close();
303 if(logData != null) {
304 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
305 WalrusBucketLogger.getInstance().addLogEntry(logData);
306 }
307 }
308 });
309 } catch(Exception ex) {
310 LOG.error(ex, ex);
311 }
312 }
313
314 public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long start, long end, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
315 try {
316 Channel channel = request.getChannel();
317 RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
318 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
319 if(etag != null)
320 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
321 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
322 if(contentDisposition != null)
323 httpResponse.addHeader("Content-Disposition", contentDisposition);
324 final ChunkedInput file;
325 isCompressed = isCompressed == null ? false : isCompressed;
326 if(isCompressed) {
327 file = new CompressedChunkedFile(raf, start, end, (int)Math.min((end - start), 8192));
328 } else {
329 file = new ChunkedDataFile(raf, start, end, (int)Math.min((end - start), 8192));
330 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf((end - start)));
331 }
332 httpResponse.addHeader("Content-Range", start + "-" + end + "/" + size);
333 if(logData != null) {
334 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
335 logData.setBytesSent(size);
336 }
337 if(versionId != null) {
338 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
339 }
340 channel.write(httpResponse);
341 channel.write(file).addListener(new ChannelFutureListener( ) {
342 @Override public void operationComplete( ChannelFuture future ) throws Exception {
343 Contexts.clear(request.getCorrelationId());
344 file.close();
345 if(logData != null) {
346 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
347 WalrusBucketLogger.getInstance().addLogEntry(logData);
348 }
349 }
350 });
351 } catch(Exception ex) {
352 LOG.error(ex, ex);
353 }
354 }
355
356 public void sendHeaders(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, Long size, String etag,
357 String lastModified, String contentType, String contentDisposition, String versionId, final BucketLogData logData) {
358 Channel channel = request.getChannel();
359 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
360 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
361 if(etag != null)
362 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
363 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
364 if(contentDisposition != null)
365 httpResponse.addHeader("Content-Disposition", contentDisposition);
366 if(versionId != null) {
367 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
368 }
369 if(logData != null) {
370 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
371 }
372 channel.write(httpResponse).addListener(new ChannelFutureListener( ) {
373 @Override public void operationComplete( ChannelFuture future ) throws Exception {
374 Contexts.clear(request.getCorrelationId());
375 if(logData != null) {
376 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
377 WalrusBucketLogger.getInstance().addLogEntry(logData);
378 }
379 }
380 });
381 }
382
383 private String removeLoopback(String loDevName) throws ExecutionException {
384 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-d", loDevName});
385 }
386
387 private int losetup(String absoluteFileName, String loDevName) {
388 try
389 {
390 Runtime rt = Runtime.getRuntime();
391 Process proc = rt.exec(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", loDevName, absoluteFileName});
392 StreamConsumer error = new StreamConsumer(proc.getErrorStream());
393 StreamConsumer output = new StreamConsumer(proc.getInputStream());
394 error.start();
395 output.start();
396 int errorCode = proc.waitFor();
397 output.join();
398 LOG.info("losetup " + loDevName + " " + absoluteFileName);
399 LOG.info(output.getReturnValue());
400 LOG.info(error.getReturnValue());
401 return errorCode;
402 } catch (Throwable t) {
403 LOG.error(t);
404 }
405 return -1;
406 }
407
408 private String findFreeLoopback() throws ExecutionException {
409 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-f"}).replaceAll("\n", "");
410 }
411
412 private String removeLogicalVolume(String lvName) throws ExecutionException {
413 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvremove", "-f", lvName});
414 }
415
416 private String reduceVolumeGroup(String vgName, String pvName) throws ExecutionException {
417 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgreduce", vgName, pvName});
418 }
419
420 private String removePhysicalVolume(String loDevName) throws ExecutionException {
421 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "pvremove", loDevName});
422 }
423
424 private String createVolumeFromLv(String lvName, String volumeKey) throws ExecutionException {
425 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "dd", "if=" + lvName, "of=" + volumeKey, "bs=1M"});
426 }
427
428 private String enableLogicalVolume(String lvName) throws ExecutionException {
429 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-ay", lvName});
430 }
431
432 private String disableLogicalVolume(String lvName) throws ExecutionException {
433 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-an", lvName});
434 }
435
436 private String removeVolumeGroup(String vgName) throws ExecutionException {
437 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgremove", vgName});
438 }
439
440 private String getLvmVersion() throws ExecutionException {
441 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvm", "version"});
442 }
443
444
445
446 public String createLoopback(String fileName) throws EucalyptusCloudException, ExecutionException {
447 int number_of_retries = 0;
448 int status = -1;
449 String loDevName;
450 do {
451 loDevName = findFreeLoopback();
452 if(loDevName.length() > 0) {
453 status = losetup(fileName, loDevName);
454 }
455 if(number_of_retries++ >= MAX_LOOP_DEVICES)
456 break;
457 } while(status != 0);
458
459 if(status != 0) {
460 throw new EucalyptusCloudException("Could not create loopback device for " + fileName +
461 ". Please check the max loop value and permissions");
462 }
463 return loDevName;
464 }
465}
4660
=== removed file 'clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java'
--- clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 2009-10-02 17:47:47 +0000
+++ clc/modules/storage-common/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 1970-01-01 00:00:00 +0000
@@ -1,108 +0,0 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage.fs;
67
68import org.apache.log4j.Logger;
69
70import java.io.FileNotFoundException;
71import java.io.FileOutputStream;
72import java.io.IOException;
73import java.nio.ByteBuffer;
74
75public class FileWriter extends FileIO {
76
77 private static Logger LOG = Logger.getLogger(FileWriter.class);
78
79 public FileWriter(String filename) throws Exception {
80 try {
81 channel = new FileOutputStream(filename).getChannel();
82 } catch( FileNotFoundException ex) {
83 LOG.error(ex);
84 throw ex;
85 }
86 }
87
88 public int read(long offset) throws IOException {
89 return -1;
90 }
91
92 public void write(byte[] bytes) throws IOException {
93 ByteBuffer buffer = ByteBuffer.wrap(bytes);
94 channel.write(buffer);
95 }
96
97 public ByteBuffer getBuffer() {
98 return null;
99 }
100
101 public void finish() {
102 try {
103 channel.close();
104 } catch(IOException ex) {
105 LOG.error(ex);
106 }
107 }
108}
1090
=== added directory 'clc/modules/storage-fs'
=== added file 'clc/modules/storage-fs/build.xml'
--- clc/modules/storage-fs/build.xml 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-fs/build.xml 2010-11-15 08:38:14 +0000
@@ -0,0 +1,68 @@
1<!--
2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 ~ Copyright (c) 2009 Eucalyptus Systems, Inc.
4 ~
5 ~ This program is free software: you can redistribute it and/or modify
6 ~ it under the terms of the GNU General Public License as published by
7 ~ the Free Software Foundation, only version 3 of the License.
8 ~
9 ~
10 ~ This file is distributed in the hope that it will be useful, but WITHOUT
11 ~ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
12 ~ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
13 ~ for more details.
14 ~
15 ~ You should have received a copy of the GNU General Public License along
16 ~ with this program. If not, see http://www.gnu.org/licenses/.
17 ~
18 ~ Please contact Eucalyptus Systems, Inc., 130 Castilian
19 ~ Dr., Goleta, CA 93101 USA or visit http://www.eucalyptus.com/licenses/
20 ~ if you need additional information or have any questions.
21 ~
22 ~ This file may incorporate work covered under the following copyright and
23 ~ permission notice:
24 ~
25 ~ Software License Agreement (BSD License)
26 ~
27 ~ Copyright (c) 2008, Regents of the University of California
28 ~ All rights reserved.
29 ~
30 ~ Redistribution and use of this software in source and binary forms, with
31 ~ or without modification, are permitted provided that the following
32 ~ conditions are met:
33 ~
34 ~ Redistributions of source code must retain the above copyright notice,
35 ~ this list of conditions and the following disclaimer.
36 ~
37 ~ Redistributions in binary form must reproduce the above copyright
38 ~ notice, this list of conditions and the following disclaimer in the
39 ~ documentation and/or other materials provided with the distribution.
40 ~
41 ~ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
42 ~ IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
43 ~ TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
44 ~ PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
45 ~ OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
46 ~ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
47 ~ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
48 ~ PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
49 ~ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
50 ~ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
51 ~ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
52 ~ THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
53 ~ LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
54 ~ SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
55 ~ IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
56 ~ BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
57 ~ THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
58 ~ OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
59 ~ WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
60 ~ ANY SUCH LICENSES OR RIGHTS.
61 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
62 ~ Author: Sunil Soman sunils@cs.ucsb.edu
63 -->
64
65<project name="eucalyptus-storage-fs" basedir=".">
66 <import file="../module-inc.xml"/>
67</project>
68
069
=== added directory 'clc/modules/storage-fs/conf'
=== added directory 'clc/modules/storage-fs/src'
=== added directory 'clc/modules/storage-fs/src/main'
=== added directory 'clc/modules/storage-fs/src/main/java'
=== added directory 'clc/modules/storage-fs/src/main/java/edu'
=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb'
=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus'
=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud'
=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws'
=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java'
--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedDataFile.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,77 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package edu.ucsb.eucalyptus.cloud.ws;
62
63import java.io.IOException;
64import java.io.RandomAccessFile;
65
66import org.jboss.netty.handler.stream.ChunkedFile;
67
68import edu.ucsb.eucalyptus.constants.IsData;
69
70public class ChunkedDataFile extends ChunkedFile implements IsData {
71
72 public ChunkedDataFile(RandomAccessFile file, long offset, long length,
73 int chunkSize) throws IOException {
74 super(file, offset, length, chunkSize);
75 }
76}
77
078
=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java'
--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedFile.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,120 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package edu.ucsb.eucalyptus.cloud.ws;
62
63import java.io.ByteArrayOutputStream;
64import java.io.RandomAccessFile;
65import java.util.zip.GZIPOutputStream;
66
67import org.apache.log4j.Logger;
68import org.jboss.netty.buffer.ChannelBuffers;
69import org.jboss.netty.handler.stream.ChunkedInput;
70
71import edu.ucsb.eucalyptus.constants.IsData;
72
73public class CompressedChunkedFile implements ChunkedInput, IsData {
74 private Logger LOG = Logger.getLogger( CompressedChunkedFile.class );
75 private RandomAccessFile file;
76 private long offset;
77 private int CHUNK_SIZE = 8192;
78 private long fileLength;
79
80 public CompressedChunkedFile(RandomAccessFile file, long fileLength) {
81 this.file = file;
82 this.offset = 0;
83 this.fileLength = fileLength;
84 }
85
86 public CompressedChunkedFile(RandomAccessFile file, long start, long end, int chunkSize) {
87 this.file = file;
88 this.offset = start;
89 this.fileLength = end;
90 CHUNK_SIZE = chunkSize;
91 }
92
93 @Override
94 public void close() throws Exception {
95 file.close();
96 }
97
98 @Override
99 public boolean hasNextChunk() throws Exception {
100 return offset < fileLength && file.getChannel().isOpen();
101 }
102
103 @Override
104 public Object nextChunk() throws Exception {
105 long offset = this.offset;
106 if (offset >= fileLength) {
107 return null;
108 }
109 int chunkSize = (int) Math.min(CHUNK_SIZE, fileLength - offset);
110 byte[] chunk = new byte[chunkSize];
111 file.readFully(chunk);
112 this.offset = offset + chunkSize;
113 ByteArrayOutputStream out = new ByteArrayOutputStream();
114 GZIPOutputStream zip = new GZIPOutputStream(out);
115 zip.write(chunk);
116 zip.close();
117 return ChannelBuffers.wrappedBuffer(out.toByteArray());
118 }
119}
120
0121
=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage'
=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java'
--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,71 @@
1/*******************************************************************************
2*Copyright (c) 2010 Takuya ASADA
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61
62package edu.ucsb.eucalyptus.storage;
63
64import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
65import edu.ucsb.eucalyptus.storage.fs.FileSystemStorageManager;
66
67public class StorageManagerFactory {
68 public static StorageManager getStorageManager(WalrusInfo walrusInfo) {
69 return new FileSystemStorageManager(walrusInfo);
70 }
71}
072
=== added directory 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs'
=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java'
--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileReader.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,114 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage.fs;
67
68import edu.ucsb.eucalyptus.storage.StorageIO;
69
70import org.apache.log4j.Logger;
71
72import java.io.FileInputStream;
73import java.io.FileNotFoundException;
74import java.io.IOException;
75import java.nio.ByteBuffer;
76import java.nio.channels.FileChannel;
77
78public class FileReader extends StorageIO {
79
80 private static Logger LOG = Logger.getLogger(FileReader.class);
81 private ByteBuffer buffer;
82 protected FileChannel channel;
83
84 public FileReader(String filename) throws Exception {
85 try {
86 channel = new FileInputStream(filename).getChannel();
87 buffer = ByteBuffer.allocate(102400/*TODO: NEIL WalrusQueryDispatcher.DATA_MESSAGE_SIZE */);
88 } catch( FileNotFoundException ex) {
89 LOG.error(ex);
90 throw ex;
91 }
92 }
93
94 public int read(long offset) throws IOException {
95 buffer.clear();
96 int bytesRead = channel.read(buffer, offset);
97 buffer.flip();
98 return bytesRead;
99 }
100
101 public void write(byte[] bytes) throws IOException {}
102
103 public ByteBuffer getBuffer() {
104 return buffer;
105 }
106
107 public void finish() {
108 try {
109 channel.close();
110 } catch(IOException ex) {
111 LOG.error(ex);
112 }
113 }
114}
0115
=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java'
--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileSystemStorageManager.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,468 @@
1/*******************************************************************************
2 *Copyright (c) 2009 Eucalyptus Systems, Inc.
3 *
4 * This program is free software: you can redistribute it and/or modify
5 * it under the terms of the GNU General Public License as published by
6 * the Free Software Foundation, only version 3 of the License.
7 *
8 *
9 * This file is distributed in the hope that it will be useful, but WITHOUT
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12 * for more details.
13 *
14 * You should have received a copy of the GNU General Public License along
15 * with this program. If not, see <http://www.gnu.org/licenses/>.
16 *
17 * Please contact Eucalyptus Systems, Inc., 130 Castilian
18 * Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19 * if you need additional information or have any questions.
20 *
21 * This file may incorporate work covered under the following copyright and
22 * permission notice:
23 *
24 * Software License Agreement (BSD License)
25 *
26 * Copyright (c) 2008, Regents of the University of California
27 * All rights reserved.
28 *
29 * Redistribution and use of this software in source and binary forms, with
30 * or without modification, are permitted provided that the following
31 * conditions are met:
32 *
33 * Redistributions of source code must retain the above copyright notice,
34 * this list of conditions and the following disclaimer.
35 *
36 * Redistributions in binary form must reproduce the above copyright
37 * notice, this list of conditions and the following disclaimer in the
38 * documentation and/or other materials provided with the distribution.
39 *
40 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42 * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43 * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44 * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51 * THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52 * LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53 * SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54 * IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55 * BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56 * THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57 * OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58 * WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59 * ANY SUCH LICENSES OR RIGHTS.
60 *******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage.fs;
67
68import com.eucalyptus.auth.util.Hashes;
69import com.eucalyptus.context.Contexts;
70import com.eucalyptus.http.MappingHttpResponse;
71import com.eucalyptus.util.EucalyptusCloudException;
72import com.eucalyptus.util.ExecutionException;
73import com.eucalyptus.util.WalrusProperties;
74import com.eucalyptus.ws.util.ChannelUtil;
75import com.eucalyptus.ws.util.WalrusBucketLogger;
76
77import edu.ucsb.eucalyptus.cloud.BucketLogData;
78import edu.ucsb.eucalyptus.cloud.ws.CompressedChunkedFile;
79import edu.ucsb.eucalyptus.cloud.ws.ChunkedDataFile;
80import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
81import edu.ucsb.eucalyptus.storage.StorageManager;
82import edu.ucsb.eucalyptus.storage.StorageIO;
83import edu.ucsb.eucalyptus.util.StreamConsumer;
84import edu.ucsb.eucalyptus.util.SystemUtil;
85
86import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
87
88import org.apache.log4j.Logger;
89import org.jboss.netty.channel.Channel;
90import org.jboss.netty.channel.ChannelFuture;
91import org.jboss.netty.channel.ChannelFutureListener;
92import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
93import org.jboss.netty.handler.codec.http.HttpHeaders;
94import org.jboss.netty.handler.stream.ChunkedInput;
95
96import java.io.*;
97import java.nio.channels.FileChannel;
98import java.util.ArrayList;
99import java.util.List;
100
101public class FileSystemStorageManager implements StorageManager {
102
103 public static final String FILE_SEPARATOR = "/";
104 public static final String lvmRootDirectory = "/dev";
105 private static boolean initialized = false;
106 private static String eucaHome = "/opt/eucalyptus";
107 public static final String EUCA_ROOT_WRAPPER = "/usr/lib/eucalyptus/euca_rootwrap";
108 public static final int MAX_LOOP_DEVICES = 256;
109 private static Logger LOG = Logger.getLogger(FileSystemStorageManager.class);
110
111 private String rootDirectory;
112 public FileSystemStorageManager(WalrusInfo walrusInfo) {
113 this.rootDirectory = walrusInfo.getStorageDir();
114 }
115
116 public void checkPreconditions() throws EucalyptusCloudException {
117 try {
118 String eucaHomeDir = System.getProperty("euca.home");
119 if(eucaHomeDir == null) {
120 throw new EucalyptusCloudException("euca.home not set");
121 }
122 eucaHome = eucaHomeDir;
123 if(!new File(eucaHome + EUCA_ROOT_WRAPPER).exists()) {
124 throw new EucalyptusCloudException("root wrapper (euca_rootwrap) does not exist");
125 }
126 String returnValue = getLvmVersion();
127 if(returnValue.length() == 0) {
128 throw new EucalyptusCloudException("Is lvm installed?");
129 } else {
130 LOG.info(returnValue);
131 }
132 } catch(ExecutionException ex) {
133 String error = "Unable to run command: " + ex.getMessage();
134 LOG.error(error);
135 throw new EucalyptusCloudException(error);
136 }
137 }
138
139 public void setRootDirectory(String rootDirectory) {
140 this.rootDirectory = rootDirectory;
141 }
142
143 public boolean bucketExists(String bucket) {
144 return new File (rootDirectory + FILE_SEPARATOR + bucket).exists();
145 }
146
147 public boolean objectExists(String bucket, String object) {
148 return new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object).exists();
149 }
150
151 public void createBucket(String bucket) throws IOException {
152 File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
153 if(!bukkit.exists()) {
154 if(!bukkit.mkdirs()) {
155 throw new IOException("Unable to create bucket: " + bucket);
156 }
157 }
158 }
159
160 public long getSize(String bucket, String object) {
161 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
162 if(objectFile.exists())
163 return objectFile.length();
164 return -1;
165 }
166
167 public void deleteBucket(String bucket) throws IOException {
168 File bukkit = new File (rootDirectory + FILE_SEPARATOR + bucket);
169 if(!bukkit.delete()) {
170 throw new IOException("Unable to delete bucket: " + bucket);
171 }
172 }
173
174 public void createObject(String bucket, String object) throws IOException {
175 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
176 if (!objectFile.exists()) {
177 if (!objectFile.createNewFile()) {
178 throw new IOException("Unable to create: " + objectFile.getAbsolutePath());
179 }
180 }
181 }
182
183 public StorageIO prepareForRead(String bucket, String object) throws Exception {
184 return new FileReader(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
185 }
186
187 public StorageIO prepareForWrite(String bucket, String object) throws Exception {
188 return new FileWriter(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
189 }
190
191 public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException {
192 return readObject(rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object, bytes, offset);
193 }
194
195 public int readObject(String path, byte[] bytes, long offset) throws IOException {
196 File objectFile = new File (path);
197 if (!objectFile.exists()) {
198 throw new IOException("Unable to read: " + path);
199 }
200 BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream(objectFile));
201 if (offset > 0) {
202 inputStream.skip(offset);
203 }
204 int bytesRead = inputStream.read(bytes);
205 inputStream.close();
206 return bytesRead;
207 }
208
209 public void deleteObject(String bucket, String object) throws IOException {
210 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
211 if (objectFile.exists()) {
212 if(!objectFile.delete()) {
213 throw new IOException("Unable to delete: " + objectFile.getAbsolutePath());
214 }
215 }
216 }
217
218 public void deleteAbsoluteObject(String object) throws IOException {
219 File objectFile = new File (object);
220 if (objectFile.exists()) {
221 if(!objectFile.delete()) {
222 throw new IOException("Unable to delete: " + object);
223 }
224 }
225 }
226
227 public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException {
228 File objectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object);
229 if (!objectFile.exists()) {
230 objectFile.createNewFile();
231 }
232 BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(objectFile, append));
233 outputStream.write(base64Data);
234 outputStream.close();
235 }
236
237 public void renameObject(String bucket, String oldName, String newName) throws IOException {
238 File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + oldName);
239 File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + newName);
240 if(oldObjectFile.exists()) {
241 if (!oldObjectFile.renameTo(newObjectFile)) {
242 throw new IOException("Unable to rename " + oldObjectFile.getAbsolutePath() + " to " + newObjectFile.getAbsolutePath());
243 }
244 }
245 }
246
247 public void copyObject(String sourceBucket, String sourceObject, String destinationBucket, String destinationObject) throws IOException {
248 File oldObjectFile = new File (rootDirectory + FILE_SEPARATOR + sourceBucket + FILE_SEPARATOR + sourceObject);
249 File newObjectFile = new File (rootDirectory + FILE_SEPARATOR + destinationBucket + FILE_SEPARATOR + destinationObject);
250 if(!oldObjectFile.equals(newObjectFile)) {
251 FileInputStream fileInputStream = new FileInputStream(oldObjectFile);
252 FileChannel fileIn = fileInputStream.getChannel();
253 FileOutputStream fileOutputStream = new FileOutputStream(newObjectFile);
254 FileChannel fileOut = fileOutputStream.getChannel();
255 fileIn.transferTo(0, fileIn.size(), fileOut);
256 fileIn.close();
257 fileInputStream.close();
258 fileOut.close();
259 fileOutputStream.close();
260 }
261 }
262
263 public String getObjectPath(String bucket, String object) {
264 return rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
265 }
266
267 public long getObjectSize(String bucket, String object) {
268 String absoluteObjectPath = rootDirectory + FILE_SEPARATOR + bucket + FILE_SEPARATOR + object;
269
270 File objectFile = new File(absoluteObjectPath);
271 if(objectFile.exists())
272 return objectFile.length();
273 return -1;
274 }
275
276 public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
277 try {
278 Channel channel = request.getChannel();
279 RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
280 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
281 if(etag != null)
282 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
283 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
284 if(contentDisposition != null)
285 httpResponse.addHeader("Content-Disposition", contentDisposition);
286 final ChunkedInput file;
287 isCompressed = isCompressed == null ? false : isCompressed;
288 if(isCompressed) {
289 file = new CompressedChunkedFile(raf, size);
290 } else {
291 file = new ChunkedDataFile(raf, 0, size, 8192);
292 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
293 }
294 if(logData != null) {
295 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
296 logData.setBytesSent(size);
297 }
298 if(versionId != null) {
299 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
300 }
301 channel.write(httpResponse);
302 channel.write(file).addListener(new ChannelFutureListener( ) {
303 @Override public void operationComplete( ChannelFuture future ) throws Exception {
304 Contexts.clear(request.getCorrelationId());
305 file.close();
306 if(logData != null) {
307 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
308 WalrusBucketLogger.getInstance().addLogEntry(logData);
309 }
310 }
311 });
312 } catch(Exception ex) {
313 LOG.error(ex, ex);
314 }
315 }
316
317 public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long start, long end, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
318 try {
319 Channel channel = request.getChannel();
320 RandomAccessFile raf = new RandomAccessFile(new File(getObjectPath(bucketName, objectName)), "r");
321 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
322 if(etag != null)
323 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
324 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
325 if(contentDisposition != null)
326 httpResponse.addHeader("Content-Disposition", contentDisposition);
327 final ChunkedInput file;
328 isCompressed = isCompressed == null ? false : isCompressed;
329 if(isCompressed) {
330 file = new CompressedChunkedFile(raf, start, end, (int)Math.min((end - start), 8192));
331 } else {
332 file = new ChunkedDataFile(raf, start, end, (int)Math.min((end - start), 8192));
333 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf((end - start)));
334 }
335 httpResponse.addHeader("Content-Range", start + "-" + end + "/" + size);
336 if(logData != null) {
337 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
338 logData.setBytesSent(size);
339 }
340 if(versionId != null) {
341 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
342 }
343 channel.write(httpResponse);
344 channel.write(file).addListener(new ChannelFutureListener( ) {
345 @Override public void operationComplete( ChannelFuture future ) throws Exception {
346 Contexts.clear(request.getCorrelationId());
347 file.close();
348 if(logData != null) {
349 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
350 WalrusBucketLogger.getInstance().addLogEntry(logData);
351 }
352 }
353 });
354 } catch(Exception ex) {
355 LOG.error(ex, ex);
356 }
357 }
358
359 public void sendHeaders(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, Long size, String etag,
360 String lastModified, String contentType, String contentDisposition, String versionId, final BucketLogData logData) {
361 Channel channel = request.getChannel();
362 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
363 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
364 if(etag != null)
365 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
366 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
367 if(contentDisposition != null)
368 httpResponse.addHeader("Content-Disposition", contentDisposition);
369 if(versionId != null) {
370 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
371 }
372 if(logData != null) {
373 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
374 }
375 channel.write(httpResponse).addListener(new ChannelFutureListener( ) {
376 @Override public void operationComplete( ChannelFuture future ) throws Exception {
377 Contexts.clear(request.getCorrelationId());
378 if(logData != null) {
379 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
380 WalrusBucketLogger.getInstance().addLogEntry(logData);
381 }
382 }
383 });
384 }
385
386 private String removeLoopback(String loDevName) throws ExecutionException {
387 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-d", loDevName});
388 }
389
390 private int losetup(String absoluteFileName, String loDevName) {
391 try
392 {
393 Runtime rt = Runtime.getRuntime();
394 Process proc = rt.exec(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", loDevName, absoluteFileName});
395 StreamConsumer error = new StreamConsumer(proc.getErrorStream());
396 StreamConsumer output = new StreamConsumer(proc.getInputStream());
397 error.start();
398 output.start();
399 int errorCode = proc.waitFor();
400 output.join();
401 LOG.info("losetup " + loDevName + " " + absoluteFileName);
402 LOG.info(output.getReturnValue());
403 LOG.info(error.getReturnValue());
404 return errorCode;
405 } catch (Throwable t) {
406 LOG.error(t);
407 }
408 return -1;
409 }
410
411 private String findFreeLoopback() throws ExecutionException {
412 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-f"}).replaceAll("\n", "");
413 }
414
415 private String removeLogicalVolume(String lvName) throws ExecutionException {
416 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvremove", "-f", lvName});
417 }
418
419 private String reduceVolumeGroup(String vgName, String pvName) throws ExecutionException {
420 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgreduce", vgName, pvName});
421 }
422
423 private String removePhysicalVolume(String loDevName) throws ExecutionException {
424 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "pvremove", loDevName});
425 }
426
427 private String createVolumeFromLv(String lvName, String volumeKey) throws ExecutionException {
428 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "dd", "if=" + lvName, "of=" + volumeKey, "bs=1M"});
429 }
430
431 private String enableLogicalVolume(String lvName) throws ExecutionException {
432 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-ay", lvName});
433 }
434
435 private String disableLogicalVolume(String lvName) throws ExecutionException {
436 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvchange", "-an", lvName});
437 }
438
439 private String removeVolumeGroup(String vgName) throws ExecutionException {
440 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgremove", vgName});
441 }
442
443 private String getLvmVersion() throws ExecutionException {
444 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvm", "version"});
445 }
446
447
448
449 public String createLoopback(String fileName) throws EucalyptusCloudException, ExecutionException {
450 int number_of_retries = 0;
451 int status = -1;
452 String loDevName;
453 do {
454 loDevName = findFreeLoopback();
455 if(loDevName.length() > 0) {
456 status = losetup(fileName, loDevName);
457 }
458 if(number_of_retries++ >= MAX_LOOP_DEVICES)
459 break;
460 } while(status != 0);
461
462 if(status != 0) {
463 throw new EucalyptusCloudException("Could not create loopback device for " + fileName +
464 ". Please check the max loop value and permissions");
465 }
466 return loDevName;
467 }
468}
0469
=== added file 'clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java'
--- clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-fs/src/main/java/edu/ucsb/eucalyptus/storage/fs/FileWriter.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,112 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61/*
62 *
63 * Author: Sunil Soman sunils@cs.ucsb.edu
64 */
65
66package edu.ucsb.eucalyptus.storage.fs;
67
68import edu.ucsb.eucalyptus.storage.StorageIO;
69
70import org.apache.log4j.Logger;
71
72import java.io.FileNotFoundException;
73import java.io.FileOutputStream;
74import java.io.IOException;
75import java.nio.ByteBuffer;
76import java.nio.channels.FileChannel;
77
78public class FileWriter extends StorageIO {
79
80 private static Logger LOG = Logger.getLogger(FileWriter.class);
81 protected FileChannel channel;
82
83 public FileWriter(String filename) throws Exception {
84 try {
85 channel = new FileOutputStream(filename).getChannel();
86 } catch( FileNotFoundException ex) {
87 LOG.error(ex);
88 throw ex;
89 }
90 }
91
92 public int read(long offset) throws IOException {
93 return -1;
94 }
95
96 public void write(byte[] bytes) throws IOException {
97 ByteBuffer buffer = ByteBuffer.wrap(bytes);
98 channel.write(buffer);
99 }
100
101 public ByteBuffer getBuffer() {
102 return null;
103 }
104
105 public void finish() {
106 try {
107 channel.close();
108 } catch(IOException ex) {
109 LOG.error(ex);
110 }
111 }
112}
0113
=== added directory 'clc/modules/storage-fs/src/main/resources'
=== added directory 'clc/modules/storage-rados'
=== added file 'clc/modules/storage-rados/build.xml'
--- clc/modules/storage-rados/build.xml 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/build.xml 2010-11-15 08:38:14 +0000
@@ -0,0 +1,68 @@
1<!--
2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 ~ Copyright (c) 2009 Eucalyptus Systems, Inc.
4 ~
5 ~ This program is free software: you can redistribute it and/or modify
6 ~ it under the terms of the GNU General Public License as published by
7 ~ the Free Software Foundation, only version 3 of the License.
8 ~
9 ~
10 ~ This file is distributed in the hope that it will be useful, but WITHOUT
11 ~ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
12 ~ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
13 ~ for more details.
14 ~
15 ~ You should have received a copy of the GNU General Public License along
16 ~ with this program. If not, see http://www.gnu.org/licenses/.
17 ~
18 ~ Please contact Eucalyptus Systems, Inc., 130 Castilian
19 ~ Dr., Goleta, CA 93101 USA or visit http://www.eucalyptus.com/licenses/
20 ~ if you need additional information or have any questions.
21 ~
22 ~ This file may incorporate work covered under the following copyright and
23 ~ permission notice:
24 ~
25 ~ Software License Agreement (BSD License)
26 ~
27 ~ Copyright (c) 2008, Regents of the University of California
28 ~ All rights reserved.
29 ~
30 ~ Redistribution and use of this software in source and binary forms, with
31 ~ or without modification, are permitted provided that the following
32 ~ conditions are met:
33 ~
34 ~ Redistributions of source code must retain the above copyright notice,
35 ~ this list of conditions and the following disclaimer.
36 ~
37 ~ Redistributions in binary form must reproduce the above copyright
38 ~ notice, this list of conditions and the following disclaimer in the
39 ~ documentation and/or other materials provided with the distribution.
40 ~
41 ~ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
42 ~ IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
43 ~ TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
44 ~ PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
45 ~ OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
46 ~ EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
47 ~ PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
48 ~ PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
49 ~ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
50 ~ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
51 ~ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
52 ~ THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
53 ~ LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
54 ~ SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
55 ~ IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
56 ~ BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
57 ~ THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
58 ~ OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
59 ~ WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
60 ~ ANY SUCH LICENSES OR RIGHTS.
61 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
62 ~ Author: Sunil Soman sunils@cs.ucsb.edu
63 -->
64
65<project name="eucalyptus-storage-rados" basedir=".">
66 <import file="../module-inc.xml"/>
67</project>
68
069
=== added directory 'clc/modules/storage-rados/conf'
=== added directory 'clc/modules/storage-rados/include'
=== added file 'clc/modules/storage-rados/include/com_dokukino_rados4j_ListCtx.h'
--- clc/modules/storage-rados/include/com_dokukino_rados4j_ListCtx.h 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/include/com_dokukino_rados4j_ListCtx.h 2010-11-15 08:38:14 +0000
@@ -0,0 +1,29 @@
1/* DO NOT EDIT THIS FILE - it is machine generated */
2#include <jni.h>
3/* Header for class com_dokukino_rados4j_ListCtx */
4
5#ifndef _Included_com_dokukino_rados4j_ListCtx
6#define _Included_com_dokukino_rados4j_ListCtx
7#ifdef __cplusplus
8extern "C" {
9#endif
10/*
11 * Class: com_dokukino_rados4j_ListCtx
12 * Method: more
13 * Signature: (I[Ljava/lang/String;)I
14 */
15JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_ListCtx_more
16 (JNIEnv *, jobject, jint, jobjectArray);
17
18/*
19 * Class: com_dokukino_rados4j_ListCtx
20 * Method: close
21 * Signature: ()V
22 */
23JNIEXPORT void JNICALL Java_com_dokukino_rados4j_ListCtx_close
24 (JNIEnv *, jobject);
25
26#ifdef __cplusplus
27}
28#endif
29#endif
030
=== added file 'clc/modules/storage-rados/include/com_dokukino_rados4j_Pool.h'
--- clc/modules/storage-rados/include/com_dokukino_rados4j_Pool.h 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/include/com_dokukino_rados4j_Pool.h 2010-11-15 08:38:14 +0000
@@ -0,0 +1,93 @@
1/* DO NOT EDIT THIS FILE - it is machine generated */
2#include <jni.h>
3/* Header for class com_dokukino_rados4j_Pool */
4
5#ifndef _Included_com_dokukino_rados4j_Pool
6#define _Included_com_dokukino_rados4j_Pool
7#ifdef __cplusplus
8extern "C" {
9#endif
10/*
11 * Class: com_dokukino_rados4j_Pool
12 * Method: delete
13 * Signature: ()I
14 */
15JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_delete
16 (JNIEnv *, jobject);
17
18/*
19 * Class: com_dokukino_rados4j_Pool
20 * Method: close
21 * Signature: ()V
22 */
23JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Pool_close
24 (JNIEnv *, jobject);
25
26/*
27 * Class: com_dokukino_rados4j_Pool
28 * Method: createObj
29 * Signature: (Ljava/lang/String;Z)I
30 */
31JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_createObj
32 (JNIEnv *, jobject, jstring, jboolean);
33
34/*
35 * Class: com_dokukino_rados4j_Pool
36 * Method: writeObj
37 * Signature: (Ljava/lang/String;J[BJ)I
38 */
39JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_writeObj
40 (JNIEnv *, jobject, jstring, jlong, jbyteArray, jlong);
41
42/*
43 * Class: com_dokukino_rados4j_Pool
44 * Method: readObj
45 * Signature: (Ljava/lang/String;J[BJ)I
46 */
47JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_readObj
48 (JNIEnv *, jobject, jstring, jlong, jbyteArray, jlong);
49
50/*
51 * Class: com_dokukino_rados4j_Pool
52 * Method: renameObj
53 * Signature: (Ljava/lang/String;Ljava/lang/String;)I
54 */
55JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_renameObj
56 (JNIEnv *, jobject, jstring, jstring);
57
58/*
59 * Class: com_dokukino_rados4j_Pool
60 * Method: copyObj
61 * Signature: (Ljava/lang/String;Lcom/dokukino/rados4j/Pool;Ljava/lang/String;)I
62 */
63JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_copyObj
64 (JNIEnv *, jobject, jstring, jobject, jstring);
65
66/*
67 * Class: com_dokukino_rados4j_Pool
68 * Method: removeObj
69 * Signature: (Ljava/lang/String;)I
70 */
71JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_removeObj
72 (JNIEnv *, jobject, jstring);
73
74/*
75 * Class: com_dokukino_rados4j_Pool
76 * Method: statObj
77 * Signature: (Ljava/lang/String;)Lcom/dokukino/rados4j/Stat;
78 */
79JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_statObj
80 (JNIEnv *, jobject, jstring);
81
82/*
83 * Class: com_dokukino_rados4j_Pool
84 * Method: openList
85 * Signature: ()Lcom/dokukino/rados4j/ListCtx;
86 */
87JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_openList
88 (JNIEnv *, jobject);
89
90#ifdef __cplusplus
91}
92#endif
93#endif
094
=== added file 'clc/modules/storage-rados/include/com_dokukino_rados4j_Rados.h'
--- clc/modules/storage-rados/include/com_dokukino_rados4j_Rados.h 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/include/com_dokukino_rados4j_Rados.h 2010-11-15 08:38:14 +0000
@@ -0,0 +1,61 @@
1/* DO NOT EDIT THIS FILE - it is machine generated */
2#include <jni.h>
3/* Header for class com_dokukino_rados4j_Rados */
4
5#ifndef _Included_com_dokukino_rados4j_Rados
6#define _Included_com_dokukino_rados4j_Rados
7#ifdef __cplusplus
8extern "C" {
9#endif
10/*
11 * Class: com_dokukino_rados4j_Rados
12 * Method: initialize
13 * Signature: ([Ljava/lang/String;)I
14 */
15JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_initialize
16 (JNIEnv *, jobject, jobjectArray);
17
18/*
19 * Class: com_dokukino_rados4j_Rados
20 * Method: shutdown
21 * Signature: ()V
22 */
23JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Rados_shutdown
24 (JNIEnv *, jobject);
25
26/*
27 * Class: com_dokukino_rados4j_Rados
28 * Method: openPool
29 * Signature: (Ljava/lang/String;)Lcom/dokukino/rados4j/Pool;
30 */
31JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Rados_openPool
32 (JNIEnv *, jobject, jstring);
33
34/*
35 * Class: com_dokukino_rados4j_Rados
36 * Method: lookupPool
37 * Signature: (Ljava/lang/String;)Z
38 */
39JNIEXPORT jboolean JNICALL Java_com_dokukino_rados4j_Rados_lookupPool
40 (JNIEnv *, jobject, jstring);
41
42/*
43 * Class: com_dokukino_rados4j_Rados
44 * Method: listPools
45 * Signature: ([Ljava/lang/String;)I
46 */
47JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_listPools
48 (JNIEnv *, jobject, jobjectArray);
49
50/*
51 * Class: com_dokukino_rados4j_Rados
52 * Method: createPool
53 * Signature: (Ljava/lang/String;J)I
54 */
55JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_createPool
56 (JNIEnv *, jobject, jstring, jlong);
57
58#ifdef __cplusplus
59}
60#endif
61#endif
062
=== added directory 'clc/modules/storage-rados/lib'
=== added directory 'clc/modules/storage-rados/native'
=== added file 'clc/modules/storage-rados/native/Makefile'
--- clc/modules/storage-rados/native/Makefile 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/native/Makefile 2010-11-15 08:38:14 +0000
@@ -0,0 +1,28 @@
1LIBDIR=../lib
2RADOS4J_SO=librados4j.so
3PLATFORM=linux
4CXXFLAGS=-g -fpic -Wall -D_FILE_OFFSET_BITS=64 -O0 -I $(JAVA_HOME)/include -I $(JAVA_HOME)/include/$(PLATFORM) -I. -I../include -I$(RADOS_HOME)/include -L$(RADOS_HOME)/lib -lcrypto -lrados -lcrush
5WARN=-Wall -Wno-unused-variable
6
7.PHONY: all clean install
8
9all: $(LIBDIR)/$(RADOS4J_SO)
10
11.cpp.o:
12 g++ -c $(WARN) $(CXXFLAGS) $<
13
14$(LIBDIR)/$(RADOS4J_SO):
15 mkdir -p $(LIBDIR)
16 gcc -shared $(CXXFLAGS) -o $(LIBDIR)/$(RADOS4J_SO) *.cpp
17
18clean:
19 rm -f $(LIBDIR)/$(RADOS4J_SO) *.o *~* *#*
20
21distclean: clean
22
23install:
24 mkdir -p $(DESTDIR)$(EUCALYPTUS)/usr/lib/eucalyptus/
25 cp $(LIBDIR)/$(RADOS4J_SO) $(DESTDIR)$(EUCALYPTUS)/usr/lib/eucalyptus/
26
27uninstall:
28 rm -r $(DESTDIR)$(EUCALYPTUS)/usr/lib/eucalyptus/$(RADOS4J_SO)
029
=== added file 'clc/modules/storage-rados/native/com_dokukino_rados4j_ListCtx.cpp'
--- clc/modules/storage-rados/native/com_dokukino_rados4j_ListCtx.cpp 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/native/com_dokukino_rados4j_ListCtx.cpp 2010-11-15 08:38:14 +0000
@@ -0,0 +1,109 @@
1#include "com_dokukino_rados4j_ListCtx.h"
2#include <rados/librados.hpp>
3#include <list>
4
5jclass list_ctx_class = NULL;
6jmethodID list_ctx_constructor = NULL;
7jfieldID list_ctx_instance_ptr = NULL, list_ctx_rados_ptr = NULL;
8
9int
10list_ctx_field_initialize(JNIEnv *env)
11{
12 jclass clsj = env->FindClass("Lcom/dokukino/rados4j/ListCtx;");
13 if (!clsj)
14 return -1;
15 list_ctx_class = reinterpret_cast<jclass>(env->NewGlobalRef(clsj));
16 env->DeleteLocalRef(clsj);
17 list_ctx_constructor = env->GetMethodID(list_ctx_class, "<init>", "()V");
18 if (!list_ctx_constructor)
19 return -1;
20 list_ctx_instance_ptr = env->GetFieldID(list_ctx_class, "instancePtr", "J");
21 if (!list_ctx_instance_ptr)
22 return -1;
23 list_ctx_rados_ptr = env->GetFieldID(list_ctx_class, "radosPtr", "J");
24 if (!list_ctx_rados_ptr)
25 return -1;
26 return 0;
27}
28
29static inline void
30list_ctx_set_instance_ptr(JNIEnv *env, jobject obj, librados::Rados::ListCtx *ptr)
31{
32 env->SetLongField(obj, list_ctx_instance_ptr, reinterpret_cast<long>(ptr));
33}
34
35static inline librados::Rados::ListCtx *
36list_ctx_get_instance_ptr(JNIEnv *env, jobject obj)
37{
38 jlong ptr = env->GetLongField(obj, list_ctx_instance_ptr);
39 return reinterpret_cast<librados::Rados::ListCtx *>(ptr);
40}
41
42static inline librados::Rados *
43list_ctx_get_rados_ptr(JNIEnv *env, jobject obj)
44{
45 jlong ptr = env->GetLongField(obj, list_ctx_rados_ptr);
46 return reinterpret_cast<librados::Rados *>(ptr);
47}
48
49extern "C"
50{
51 /*
52 * Class: com_dokukino_rados4j_ListCtx
53 * Method: more
54 * Signature: (I[Ljava/lang/String;)I
55 */
56 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_ListCtx_more
57 (JNIEnv *env, jobject obj, jint max, jobjectArray entries)
58 {
59 librados::Rados::ListCtx *ctx;
60 librados::Rados *rados;
61 std::list<std::string> entries_list;
62 jint r;
63
64 if (!entries)
65 return -1;
66 ctx = list_ctx_get_instance_ptr(env, obj);
67 if (!ctx)
68 return -1;
69 rados = list_ctx_get_rados_ptr(env, obj);
70 if (!rados)
71 return -1;
72 r = rados->list_objects_more(*ctx, max, entries_list);
73 if (r < 0)
74 return r;
75 int i = 0;
76 const int len = env->GetArrayLength(entries);
77 for (std::list<std::string>::iterator iter = entries_list.begin();
78 i < len && iter != entries_list.end(); ++iter, ++i)
79 {
80 jstring entry = env->NewStringUTF((*iter).c_str());
81 env->SetObjectArrayElement(entries, i, entry);
82 }
83
84 return r;
85 }
86
87 /*
88 * Class: com_dokukino_rados4j_ListCtx
89 * Method: close
90 * Signature: ()V
91 */
92 JNIEXPORT void JNICALL Java_com_dokukino_rados4j_ListCtx_close
93 (JNIEnv *env, jobject obj)
94 {
95 librados::Rados::ListCtx *ctx;
96 librados::Rados *rados;
97
98 ctx = list_ctx_get_instance_ptr(env, obj);
99 if (!ctx)
100 return;
101 rados = list_ctx_get_rados_ptr(env, obj);
102 if (!rados)
103 return;
104 rados->list_objects_close(*ctx);
105 delete ctx;
106 list_ctx_set_instance_ptr(env, obj, NULL);
107 }
108
109}
0110
=== added file 'clc/modules/storage-rados/native/com_dokukino_rados4j_Pool.cpp'
--- clc/modules/storage-rados/native/com_dokukino_rados4j_Pool.cpp 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/native/com_dokukino_rados4j_Pool.cpp 2010-11-15 08:38:14 +0000
@@ -0,0 +1,484 @@
1#include "com_dokukino_rados4j_Pool.h"
2#include <rados/librados.hpp>
3#include <iostream>
4
5jclass pool_class = NULL, stat_class = NULL;
6jfieldID pool_instance_ptr = NULL, pool_rados_ptr = NULL, stat_psize = NULL, stat_pmtime = NULL;
7jmethodID pool_constructor = NULL, stat_constructor = NULL;
8
9extern jclass list_ctx_class;
10extern jmethodID list_ctx_constructor;
11extern jfieldID list_ctx_instance_ptr, list_ctx_rados_ptr;
12
13int
14pool_field_initialize(JNIEnv *env)
15{
16 jclass clsj = env->FindClass("Lcom/dokukino/rados4j/Pool;");
17 if (!clsj)
18 return -1;
19 pool_class = reinterpret_cast<jclass>(env->NewGlobalRef(clsj));
20 env->DeleteLocalRef(clsj);
21
22 pool_constructor = env->GetMethodID(pool_class, "<init>", "()V");
23 if (!pool_constructor)
24 return -1;
25 pool_instance_ptr = env->GetFieldID(pool_class, "instancePtr", "J");
26 if (!pool_instance_ptr)
27 return -1;
28 pool_rados_ptr = env->GetFieldID(pool_class, "radosPtr", "J");
29 if (!pool_rados_ptr)
30 return -1;
31 return 0;
32}
33
34int
35stat_field_initialize(JNIEnv *env)
36{
37 jclass clsj = env->FindClass("Lcom/dokukino/rados4j/Stat;");
38 if (!clsj)
39 return -1;
40 stat_class = reinterpret_cast<jclass>(env->NewGlobalRef(clsj));
41 env->DeleteLocalRef(clsj);
42
43 stat_constructor = env->GetMethodID(stat_class, "<init>", "()V");
44 if (!stat_constructor)
45 return -1;
46 stat_psize = env->GetFieldID(stat_class, "psize", "J");
47 if (!stat_psize)
48 return -1;
49 stat_pmtime = env->GetFieldID(stat_class, "pmtime", "J");
50 if (!stat_pmtime)
51 return -1;
52 return 0;
53}
54
55static inline void
56pool_set_instance_ptr(JNIEnv *env, jobject obj, librados::pool_t pool)
57{
58 env->SetLongField(obj, pool_instance_ptr, reinterpret_cast<jlong>(pool));
59}
60
61static inline void
62pool_set_rados_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
63{
64 env->SetLongField(obj, pool_rados_ptr, reinterpret_cast<jlong>(rados));
65}
66
67static inline librados::pool_t
68pool_get_instance_ptr(JNIEnv *env, jobject obj)
69{
70 jlong ptr = env->GetLongField(obj, pool_instance_ptr);
71 return reinterpret_cast<librados::pool_t>(ptr);
72}
73
74static inline librados::Rados *
75pool_get_rados_ptr(JNIEnv *env, jobject obj)
76{
77 jlong ptr = env->GetLongField(obj, pool_rados_ptr);
78 return reinterpret_cast<librados::Rados *>(ptr);
79}
80
81static inline jobject
82pool_new_instance(JNIEnv *env)
83{
84 return env->NewObject(pool_class, pool_constructor);
85}
86
87static inline jobject
88stat_new_instance(JNIEnv *env)
89{
90 return env->NewObject(stat_class, stat_constructor);
91}
92
93static inline void
94stat_set_psize(JNIEnv *env, jobject obj, uint64_t psize)
95{
96 env->SetLongField(obj, stat_psize, static_cast<jlong>(psize));
97}
98
99static inline void
100stat_set_pmtime(JNIEnv *env, jobject obj, time_t pmtime)
101{
102 env->SetLongField(obj, stat_pmtime, static_cast<jlong>(pmtime));
103}
104
105static inline jobject
106list_ctx_new_instance(JNIEnv *env)
107{
108 return env->NewObject(list_ctx_class, list_ctx_constructor);
109}
110
111static inline void
112list_ctx_set_instance_ptr(JNIEnv *env, jobject obj, librados::Rados::ListCtx *ctx)
113{
114 env->SetLongField(obj, list_ctx_instance_ptr, reinterpret_cast<jlong>(ctx));
115}
116
117static inline void
118list_ctx_set_rados_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
119{
120 env->SetLongField(obj, list_ctx_rados_ptr, reinterpret_cast<jlong>(rados));
121}
122
123extern "C"
124{
125
126 /*
127 * Class: com_dokukino_rados4j_Pool
128 * Method: delete
129 * Signature: ()I
130 */
131 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_delete
132 (JNIEnv *env, jobject obj)
133 {
134 librados::Rados *rados;
135 librados::pool_t pool_ptr;
136
137 rados = pool_get_rados_ptr(env, obj);
138 if (!rados)
139 return -1;
140 pool_ptr = pool_get_instance_ptr(env, obj);
141 if (!pool_ptr)
142 return -1;
143 return rados->delete_pool(pool_ptr);
144 }
145
146 /*
147 * Class: com_dokukino_rados4j_Pool
148 * Method: close
149 * Signature: ()V
150 */
151 JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Pool_close
152 (JNIEnv *env, jobject obj)
153 {
154 librados::Rados *rados;
155 librados::pool_t pool_ptr;
156
157 rados = pool_get_rados_ptr(env, obj);
158 if (!rados)
159 return;
160 pool_ptr = pool_get_instance_ptr(env, obj);
161 if (!pool_ptr)
162 return;
163
164 rados->close_pool(pool_ptr);
165 pool_set_instance_ptr(env, obj, NULL);
166 }
167
168 /*
169 * Class: com_dokukino_rados4j_Pool
170 * Method: createObj
171 * Signature: (Ljava/lang/String;Z)I
172 */
173 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_createObj
174 (JNIEnv *env, jobject obj, jstring oid, jboolean exclusive)
175 {
176 librados::Rados *rados;
177 librados::pool_t pool_ptr;
178 const char *oid_char;
179 jint ret;
180
181 if (!oid)
182 return -1;
183 rados = pool_get_rados_ptr(env, obj);
184 if (!rados)
185 return -1;
186 pool_ptr = pool_get_instance_ptr(env, obj);
187 if (!pool_ptr)
188 return -1;
189 oid_char = env->GetStringUTFChars(oid, NULL);
190 if (!oid_char)
191 return -1;
192 ret = rados->create(pool_ptr, oid_char, exclusive);
193 env->ReleaseStringUTFChars(oid, oid_char);
194 return ret;
195 }
196
197 /*
198 * Class: com_dokukino_rados4j_Pool
199 * Method: writeObj
200 * Signature: (Ljava/lang/String;J[BJ)I
201 */
202 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_writeObj
203 (JNIEnv *env, jobject obj, jstring oid, jlong off, jbyteArray buf, jlong len)
204 {
205 librados::Rados *rados;
206 librados::pool_t pool_ptr;
207 librados::bufferlist bl;
208 const char *oid_char;
209 jbyte* src;
210 jint ret;
211
212 if (!oid || !buf)
213 return -1;
214
215 rados = pool_get_rados_ptr(env, obj);
216 if (!rados)
217 return -1;
218
219 pool_ptr = pool_get_instance_ptr(env, obj);
220 if (!pool_ptr)
221 return -1;
222
223 oid_char = env->GetStringUTFChars(oid, NULL);
224 if (!oid_char)
225 return -1;
226
227 src = env->GetByteArrayElements(buf, NULL);
228 bl.append(reinterpret_cast<char *>(src), len);
229 ret = rados->write(pool_ptr, oid_char, off, bl, len);
230 env->ReleaseByteArrayElements(buf, src, 0);
231 env->ReleaseStringUTFChars(oid, oid_char);
232
233 return ret;
234 }
235
236 /*
237 * Class: com_dokukino_rados4j_Pool
238 * Method: readObj
239 * Signature: (Ljava/lang/String;J[BJ)I
240 */
241 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_readObj
242 (JNIEnv *env, jobject obj, jstring oid, jlong off, jbyteArray buf, jlong len)
243 {
244 librados::Rados *rados;
245 librados::pool_t pool_ptr;
246 librados::bufferlist bl;
247 const char *oid_char;
248 jint ret;
249
250 if (!oid || !buf)
251 return -1;
252 rados = pool_get_rados_ptr(env, obj);
253 if (!rados)
254 return -1;
255 pool_ptr = pool_get_instance_ptr(env, obj);
256 if (!pool_ptr)
257 return -1;
258 oid_char = env->GetStringUTFChars(oid, NULL);
259 if (!oid_char)
260 return -1;
261
262 ret = rados->read(pool_ptr, oid_char, off, bl, len);
263
264 if (ret > 0)
265 env->SetByteArrayRegion(buf, 0, ret, reinterpret_cast<jbyte*>(bl.c_str()));
266 env->ReleaseStringUTFChars(oid, oid_char);
267
268 return ret;
269 }
270
271
272 /*
273 * Class: com_dokukino_rados4j_Pool
274 * Method: renameObj
275 * Signature: (Ljava/lang/String;Ljava/lang/String;)I
276 */
277 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_renameObj
278 (JNIEnv *env, jobject obj, jstring old_oid, jstring new_oid)
279 {
280 librados::Rados *rados;
281 librados::pool_t pool_ptr;
282 librados::bufferlist bl;
283 const char *old_oid_char, *new_oid_char;
284 jint ret;
285
286 if (!old_oid || !new_oid)
287 return -1;
288 rados = pool_get_rados_ptr(env, obj);
289 if (!rados)
290 return -1;
291 pool_ptr = pool_get_instance_ptr(env, obj);
292 if (!pool_ptr)
293 return -1;
294 old_oid_char = env->GetStringUTFChars(old_oid, NULL);
295 if (!old_oid_char)
296 return -1;
297 new_oid_char = env->GetStringUTFChars(new_oid, NULL);
298 if (!new_oid_char)
299 {
300 env->ReleaseStringUTFChars(old_oid, old_oid_char);
301 return -1;
302 }
303
304 ret = rados->read(pool_ptr, old_oid_char, 0, bl, 0);
305 if (ret < 0)
306 goto err;
307
308 ret = rados->write_full(pool_ptr, new_oid_char, bl);
309 if (ret < 0)
310 goto err;
311
312 rados->remove(pool_ptr, old_oid_char);
313err:
314 env->ReleaseStringUTFChars(old_oid, old_oid_char);
315 env->ReleaseStringUTFChars(new_oid, new_oid_char);
316 return ret;
317 }
318
319 /*
320 * Class: com_dokukino_rados4j_Pool
321 * Method: copyObj
322 * Signature: (Ljava/lang/String;Lcom/dokukino/rados4j/Pool;Ljava/lang/String;)I
323 */
324 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_copyObj
325 (JNIEnv *env, jobject obj, jstring src_oid, jobject dest_pool, jstring dest_oid)
326 {
327 librados::Rados *rados;
328 librados::pool_t src_pool_ptr, dest_pool_ptr;
329 librados::bufferlist bl;
330 const char *src_oid_char, *dest_oid_char;
331 jint ret;
332
333 if (!src_oid || !dest_pool || !dest_oid)
334 return -1;
335 rados = pool_get_rados_ptr(env, obj);
336 if (!rados)
337 return -1;
338 src_pool_ptr = pool_get_instance_ptr(env, obj);
339 if (!src_pool_ptr)
340 return -1;
341 dest_pool_ptr = pool_get_instance_ptr(env, dest_pool);
342 if (!dest_pool_ptr)
343 return -1;
344 src_oid_char = env->GetStringUTFChars(src_oid, NULL);
345 if (!src_oid_char)
346 return -1;
347 dest_oid_char = env->GetStringUTFChars(dest_oid, NULL);
348 if (!dest_oid_char)
349 {
350 env->ReleaseStringUTFChars(src_oid, src_oid_char);
351 return -1;
352 }
353
354 ret = rados->read(src_pool_ptr, src_oid_char, 0, bl, 0);
355 if (ret < 0)
356 goto err;
357
358 ret = rados->write_full(dest_pool_ptr, dest_oid_char, bl);
359 if (ret < 0)
360 goto err;
361
362err:
363 env->ReleaseStringUTFChars(src_oid, src_oid_char);
364 env->ReleaseStringUTFChars(dest_oid, dest_oid_char);
365 return ret;
366 }
367
368
369 /*
370 * Class: com_dokukino_rados4j_Pool
371 * Method: removeObj
372 * Signature: (Ljava/lang/String;)I
373 */
374 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Pool_removeObj
375 (JNIEnv *env, jobject obj, jstring oid)
376 {
377 librados::Rados *rados;
378 librados::pool_t pool_ptr;
379 const char *oid_char;
380 jint ret;
381
382 if (!oid)
383 return -1;
384 rados = pool_get_rados_ptr(env, obj);
385 if (!rados)
386 return -1;
387 pool_ptr = pool_get_instance_ptr(env, obj);
388 if (!pool_ptr)
389 return -1;
390 oid_char = env->GetStringUTFChars(oid, NULL);
391 if (!oid_char)
392 return -1;
393 ret = rados->remove(pool_ptr, oid_char);
394 env->ReleaseStringUTFChars(oid, oid_char);
395
396 return ret;
397 }
398
399 /*
400 * Class: com_dokukino_rados4j_Pool
401 * Method: statObj
402 * Signature: (Ljava/lang/String;)Lcom/dokukino/rados4j/Stat;
403 */
404 JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_statObj
405 (JNIEnv *env, jobject obj, jstring oid)
406 {
407 librados::Rados *rados;
408 librados::pool_t pool_ptr;
409 const char *oid_char;
410 uint64_t psize;
411 time_t pmtime;
412 jint ret;
413
414 if (!oid)
415 return NULL;
416
417 rados = pool_get_rados_ptr(env, obj);
418 if (!rados)
419 return NULL;
420
421 pool_ptr = pool_get_instance_ptr(env, obj);
422 if (!pool_ptr)
423 return NULL;
424
425 oid_char = env->GetStringUTFChars(oid, NULL);
426 if (!oid_char)
427 return NULL;
428
429 ret = rados->stat(pool_ptr, oid_char, &psize, &pmtime);
430
431 env->ReleaseStringUTFChars(oid, oid_char);
432 if (ret < 0)
433 return NULL;
434 else
435 {
436 jobject obj = stat_new_instance(env);
437 if (!obj)
438 return NULL;
439 stat_set_psize(env, obj, psize);
440 stat_set_pmtime(env, obj, pmtime);
441 return obj;
442 }
443 }
444
445 /*
446 * Class: com_dokukino_rados4j_Pool
447 * Method: openList
448 * Signature: ()Lcom/dokukino/rados4j/ListCtx;
449 */
450 JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Pool_openList
451 (JNIEnv *env, jobject obj)
452 {
453 librados::Rados *rados;
454 librados::pool_t pool_ptr;
455 librados::Rados::ListCtx *ctx = NULL;
456 jint ret;
457
458 rados = pool_get_rados_ptr(env, obj);
459 if (!rados)
460 return NULL;
461
462 pool_ptr = pool_get_instance_ptr(env, obj);
463 if (!pool_ptr)
464 return NULL;
465
466 ctx = new librados::Rados::ListCtx();
467 ret = rados->list_objects_open(pool_ptr, ctx);
468
469 if (ret < 0)
470 {
471 delete ctx;
472 return NULL;
473 }
474 else
475 {
476 jobject obj = list_ctx_new_instance(env);
477 if (!obj)
478 return NULL;
479 list_ctx_set_instance_ptr(env, obj, ctx);
480 list_ctx_set_rados_ptr(env, obj, rados);
481 return obj;
482 }
483 }
484}
0485
=== added file 'clc/modules/storage-rados/native/com_dokukino_rados4j_Rados.cpp'
--- clc/modules/storage-rados/native/com_dokukino_rados4j_Rados.cpp 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/native/com_dokukino_rados4j_Rados.cpp 2010-11-15 08:38:14 +0000
@@ -0,0 +1,245 @@
1#include "com_dokukino_rados4j_Rados.h"
2#include <rados/librados.hpp>
3#include <iostream>
4
5static jfieldID rados_instance_ptr = NULL;
6
7extern jclass pool_class;
8extern jfieldID pool_instance_ptr, pool_rados_ptr;
9extern jmethodID pool_constructor;
10
11extern int pool_field_initialize(JNIEnv *);
12extern int stat_field_initialize(JNIEnv *);
13extern int list_ctx_field_initialize(JNIEnv *);
14
15static int
16rados_field_initialize(JNIEnv *env)
17{
18 jclass clsj = env->FindClass("Lcom/dokukino/rados4j/Rados;");
19 if (!clsj)
20 {
21 printf("findclass failed\n");
22 return -1;
23 }
24 rados_instance_ptr = env->GetFieldID(clsj, "instancePtr", "J");
25 if (!rados_instance_ptr)
26 {
27 printf("getfieldid failed\n");
28 return -1;
29 }
30 return 0;
31}
32
33static inline void
34rados_set_instance_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
35{
36 env->SetLongField(obj, rados_instance_ptr, reinterpret_cast<jlong>(rados));
37}
38
39static inline librados::Rados *
40rados_get_instance_ptr(JNIEnv *env, jobject obj)
41{
42 jlong ptr = env->GetLongField(obj, rados_instance_ptr);
43 return reinterpret_cast<librados::Rados *>(ptr);
44}
45
46static inline jobject
47pool_new_instance(JNIEnv *env)
48{
49 return env->NewObject(pool_class, pool_constructor);
50}
51
52static inline void
53pool_set_instance_ptr(JNIEnv *env, jobject obj, librados::pool_t pool)
54{
55 env->SetLongField(obj, pool_instance_ptr, reinterpret_cast<jlong>(pool));
56}
57
58static inline void
59pool_set_rados_ptr(JNIEnv *env, jobject obj, librados::Rados *rados)
60{
61 env->SetLongField(obj, pool_rados_ptr, reinterpret_cast<jlong>(rados));
62}
63
64extern "C"
65{
66 /*
67 * Class: com_dokukino_rados4j_Rados
68 * Method: initialize
69 * Signature: ([Ljava/lang/String;)I
70 */
71 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_initialize
72 (JNIEnv *env, jobject obj, jobjectArray args)
73 {
74 jint ret;
75 librados::Rados *rados;
76 ret = rados_field_initialize(env);
77 if (ret)
78 return -1;
79
80 ret = pool_field_initialize(env);
81 if (ret)
82 return -1;
83
84 ret = stat_field_initialize(env);
85 if (ret)
86 return -1;
87
88 ret = list_ctx_field_initialize(env);
89 if (ret)
90 return -1;
91
92 rados = new librados::Rados();
93 if (!rados)
94 return -1;
95
96 ret = rados->initialize(0, NULL);
97 rados_set_instance_ptr(env, obj, rados);
98 return ret;
99 }
100
101 /*
102 * Class: com_dokukino_rados4j_Rados
103 * Method: shutdown
104 * Signature: ()V
105 */
106 JNIEXPORT void JNICALL Java_com_dokukino_rados4j_Rados_shutdown
107 (JNIEnv *env, jobject obj)
108 {
109 librados::Rados *rados;
110
111 rados = rados_get_instance_ptr(env, obj);
112 if (!rados)
113 return;
114 rados->shutdown();
115 delete rados;
116 rados_set_instance_ptr(env, obj, NULL);
117 }
118
119 /*
120 * Class: com_dokukino_rados4j_Rados
121 * Method: openPool
122 * Signature: (Ljava/lang/String;Lcom/dokukino/rados4j/Pool;)I
123 */
124 JNIEXPORT jobject JNICALL Java_com_dokukino_rados4j_Rados_openPool
125 (JNIEnv *env, jobject obj, jstring name)
126 {
127 librados::Rados *rados;
128 const char *name_char;
129 librados::pool_t pool;
130 jint ret;
131
132 if (!name)
133 return NULL;
134 rados = rados_get_instance_ptr(env, obj);
135 if (!rados)
136 return NULL;
137 name_char = env->GetStringUTFChars(name, NULL);
138 if (!name_char)
139 return NULL;
140 ret = rados->open_pool(name_char, &pool);
141
142 env->ReleaseStringUTFChars(name, name_char);
143 if (ret < 0)
144 return NULL;
145 else
146 {
147 jobject obj = pool_new_instance(env);
148 if (!obj)
149 {
150 rados->close_pool(pool);
151 return NULL;
152 }
153 pool_set_instance_ptr(env, obj, pool);
154 pool_set_rados_ptr(env, obj, rados);
155 return obj;
156 }
157 }
158
159 /*
160 * Class: com_dokukino_rados4j_Rados
161 * Method: lookupPool
162 * Signature: (Ljava/lang/String;)Z
163 */
164 JNIEXPORT jboolean JNICALL Java_com_dokukino_rados4j_Rados_lookupPool
165 (JNIEnv *env, jobject obj, jstring name)
166 {
167 librados::Rados *rados;
168 const char *name_char;
169 int ret;
170
171 if (!name)
172 return -1;
173
174 rados = rados_get_instance_ptr(env, obj);
175 if (!rados)
176 return -1;
177
178 name_char = env->GetStringUTFChars(name, NULL);
179 if (!name_char)
180 return -1;
181
182 ret = rados->lookup_pool(name_char);
183 env->ReleaseStringUTFChars(name, name_char);
184
185 return (ret > 0);
186 }
187
188 /*
189 * Class: com_dokukino_rados4j_Rados
190 * Method: listPools
191 * Signature: ([Ljava/lang/String;)I
192 */
193 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_listPools
194 (JNIEnv *env, jobject obj, jobjectArray entries)
195 {
196 librados::Rados *rados;
197 std::list<std::string> entries_list;
198 int r;
199
200 if (!entries)
201 return -1;
202
203 rados = rados_get_instance_ptr(env, obj);
204 if (!rados)
205 return -1;
206
207 r = rados->list_pools(entries_list);
208 if (r < 0)
209 return r;
210
211 int i = 0;
212 const int len = env->GetArrayLength(entries);
213 for (std::list<std::string>::iterator iter = entries_list.begin();
214 i < len && iter != entries_list.end(); ++iter, ++i)
215 {
216 jstring entry = env->NewStringUTF((*iter).c_str());
217 env->SetObjectArrayElement(entries, i, entry);
218 }
219
220 return r;
221 }
222
223 /*
224 * Class: com_dokukino_rados4j_Rados
225 * Method: createPool
226 * Signature: (Ljava/lang/String;J)I
227 */
228 JNIEXPORT jint JNICALL Java_com_dokukino_rados4j_Rados_createPool
229 (JNIEnv *env, jobject obj, jstring name, jlong auid)
230 {
231 librados::Rados *rados;
232 const char *name_char;
233
234 if (!name)
235 return -1;
236 rados = rados_get_instance_ptr(env, obj);
237 if (!rados)
238 return -1;
239 name_char = env->GetStringUTFChars(name, NULL);
240 if (!name_char)
241 return -1;
242 return rados->create_pool(name_char, auid);
243 }
244
245}
0246
=== added directory 'clc/modules/storage-rados/src'
=== added directory 'clc/modules/storage-rados/src/main'
=== added directory 'clc/modules/storage-rados/src/main/java'
=== added directory 'clc/modules/storage-rados/src/main/java/com'
=== added directory 'clc/modules/storage-rados/src/main/java/com/dokukino'
=== added directory 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j'
=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/ListCtx.java'
--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/ListCtx.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/ListCtx.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,68 @@
1/*******************************************************************************
2*Copyright (c) 2010 Takuya ASADA
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package com.dokukino.rados4j;
62
63public class ListCtx {
64 private long radosPtr, instancePtr;
65
66 public native int more(int max, String[] entries);
67 public native void close();
68}
069
=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Pool.java'
--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Pool.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Pool.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,76 @@
1/*******************************************************************************
2*Copyright (c) 2010 Takuya ASADA
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package com.dokukino.rados4j;
62
63public class Pool {
64 private long radosPtr, instancePtr;
65
66 public native int delete();
67 public native void close();
68 public native int createObj(String oid, boolean exclusive);
69 public native int writeObj(String oid, long off, byte[] buf, long len);
70 public native int readObj(String oid, long off, byte[] buf, long len);
71 public native int renameObj(String oldOid, String newOid);
72 public native int copyObj(String srcOid, Pool destPool, String destOid);
73 public native int removeObj(String oid);
74 public native Stat statObj(String oid);
75 public native ListCtx openList();
76}
077
=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Rados.java'
--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Rados.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Rados.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,77 @@
1/*******************************************************************************
2*Copyright (c) 2010 Takuya ASADA
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package com.dokukino.rados4j;
62
63public class Rados {
64 private long instancePtr;
65
66 static {
67 System.loadLibrary("rados4j");
68 }
69
70 public native int initialize(String[] args);
71 public native void shutdown();
72 public native Pool openPool(String name);
73 public native boolean lookupPool(String name);
74 public native int listPools(String[] v);
75 public native int createPool(String name, long auid);
76 public int createPool(String name) { return createPool(name, 0); }
77}
078
=== added file 'clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Stat.java'
--- clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Stat.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/com/dokukino/rados4j/Stat.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,76 @@
1/*******************************************************************************
2*Copyright (c) 2010 Takuya ASADA
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package com.dokukino.rados4j;
62
63import java.util.Date;
64
65public class Stat {
66 private long psize;
67 private long pmtime;
68
69 public long getSize() {
70 return psize;
71 }
72
73 public long getMtime() {
74 return pmtime;
75 }
76}
077
=== added directory 'clc/modules/storage-rados/src/main/java/edu'
=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb'
=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus'
=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud'
=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws'
=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedRadosObject.java'
--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedRadosObject.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/ChunkedRadosObject.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,125 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package edu.ucsb.eucalyptus.cloud.ws;
62
63import java.io.ByteArrayOutputStream;
64import java.io.IOException;
65
66import org.apache.log4j.Logger;
67import org.jboss.netty.buffer.ChannelBuffers;
68import org.jboss.netty.handler.stream.ChunkedInput;
69
70import edu.ucsb.eucalyptus.constants.IsData;
71
72import com.dokukino.rados4j.*;
73
74public class ChunkedRadosObject implements ChunkedInput, IsData {
75 private Logger LOG = Logger.getLogger( ChunkedRadosObject.class );
76 private Pool pool;
77 private String object;
78 private long offset;
79 private int CHUNK_SIZE = 8192;
80 private long length;
81
82 public ChunkedRadosObject(Pool pool, String object, long length) {
83 this.pool = pool;
84 this.object = object;
85 this.offset = 0;
86 this.length = length;
87 }
88
89 public ChunkedRadosObject(Pool pool, String object, long start, long end, int chunkSize) {
90 this.pool = pool;
91 this.object = object;
92 this.offset = start;
93 this.length = end;
94 CHUNK_SIZE = chunkSize;
95 }
96
97 @Override
98 public void close() throws Exception {
99 pool.close();
100 }
101
102 @Override
103 public boolean hasNextChunk() throws Exception {
104 return offset < length;
105 }
106
107 @Override
108 public Object nextChunk() throws Exception {
109 long offset = this.offset;
110 if (offset >= length) {
111 return null;
112 }
113 int chunkSize = (int) Math.min(CHUNK_SIZE, length - offset);
114 byte[] chunk = new byte[chunkSize];
115 int ret = pool.readObj(object, offset, chunk, chunkSize);
116 if (ret < 0)
117 throw new IOException("Unable to read object: " + object);
118 this.offset = offset + chunkSize;
119 ByteArrayOutputStream out = new ByteArrayOutputStream();
120 out.write(chunk);
121 out.close();
122 return ChannelBuffers.wrappedBuffer(out.toByteArray());
123 }
124}
125
0126
=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedRadosObject.java'
--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedRadosObject.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/cloud/ws/CompressedChunkedRadosObject.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,127 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61package edu.ucsb.eucalyptus.cloud.ws;
62
63import java.io.ByteArrayOutputStream;
64import java.io.IOException;
65import java.util.zip.GZIPOutputStream;
66
67import org.apache.log4j.Logger;
68import org.jboss.netty.buffer.ChannelBuffers;
69import org.jboss.netty.handler.stream.ChunkedInput;
70
71import edu.ucsb.eucalyptus.constants.IsData;
72
73import com.dokukino.rados4j.*;
74
75public class CompressedChunkedRadosObject implements ChunkedInput, IsData {
76 private Logger LOG = Logger.getLogger( CompressedChunkedRadosObject.class );
77 private Pool pool;
78 private String object;
79 private long offset;
80 private int CHUNK_SIZE = 8192;
81 private long length;
82
83 public CompressedChunkedRadosObject(Pool pool, String object, long length) {
84 this.pool = pool;
85 this.object = object;
86 this.offset = 0;
87 this.length = length;
88 }
89
90 public CompressedChunkedRadosObject(Pool pool, String object, long start, long end, int chunkSize) {
91 this.pool = pool;
92 this.object = object;
93 this.offset = start;
94 this.length = end;
95 CHUNK_SIZE = chunkSize;
96 }
97
98 @Override
99 public void close() throws Exception {
100 pool.close();
101 }
102
103 @Override
104 public boolean hasNextChunk() throws Exception {
105 return offset < length;
106 }
107
108 @Override
109 public Object nextChunk() throws Exception {
110 long offset = this.offset;
111 if (offset >= length) {
112 return null;
113 }
114 int chunkSize = (int) Math.min(CHUNK_SIZE, length - offset);
115 byte[] chunk = new byte[chunkSize];
116 int ret = pool.readObj(object, offset, chunk, chunkSize);
117 if (ret < 0)
118 throw new IOException("Unable to read object: " + object);
119 this.offset = offset + chunkSize;
120 ByteArrayOutputStream out = new ByteArrayOutputStream();
121 GZIPOutputStream zip = new GZIPOutputStream(out);
122 zip.write(chunk);
123 zip.close();
124 return ChannelBuffers.wrappedBuffer(out.toByteArray());
125 }
126}
127
0128
=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage'
=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java'
--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/StorageManagerFactory.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,71 @@
1/*******************************************************************************
2*Copyright (c) 2010 Takuya ASADA
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61
62package edu.ucsb.eucalyptus.storage;
63
64import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
65import edu.ucsb.eucalyptus.storage.rados.RadosStorageManager;
66
67public class StorageManagerFactory {
68 public static StorageManager getStorageManager(WalrusInfo walrusInfo) {
69 return new RadosStorageManager(walrusInfo);
70 }
71}
072
=== added directory 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados'
=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosReader.java'
--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosReader.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosReader.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,118 @@
1/*******************************************************************************
2*Copyright (c) 2009 Eucalyptus Systems, Inc.
3*
4* This program is free software: you can redistribute it and/or modify
5* it under the terms of the GNU General Public License as published by
6* the Free Software Foundation, only version 3 of the License.
7*
8*
9* This file is distributed in the hope that it will be useful, but WITHOUT
10* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12* for more details.
13*
14* You should have received a copy of the GNU General Public License along
15* with this program. If not, see <http://www.gnu.org/licenses/>.
16*
17* Please contact Eucalyptus Systems, Inc., 130 Castilian
18* Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19* if you need additional information or have any questions.
20*
21* This file may incorporate work covered under the following copyright and
22* permission notice:
23*
24* Software License Agreement (BSD License)
25*
26* Copyright (c) 2008, Regents of the University of California
27* All rights reserved.
28*
29* Redistribution and use of this software in source and binary forms, with
30* or without modification, are permitted provided that the following
31* conditions are met:
32*
33* Redistributions of source code must retain the above copyright notice,
34* this list of conditions and the following disclaimer.
35*
36* Redistributions in binary form must reproduce the above copyright
37* notice, this list of conditions and the following disclaimer in the
38* documentation and/or other materials provided with the distribution.
39*
40* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43* PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44* OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51* THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52* LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53* SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54* IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55* BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56* THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57* OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58* WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59* ANY SUCH LICENSES OR RIGHTS.
60*******************************************************************************/
61
62package edu.ucsb.eucalyptus.storage.rados;
63
64import edu.ucsb.eucalyptus.storage.StorageIO;
65
66import org.apache.log4j.Logger;
67
68import java.io.FileNotFoundException;
69import java.io.IOException;
70import java.nio.ByteBuffer;
71
72import com.dokukino.rados4j.*;
73
74public class RadosReader extends StorageIO {
75
76 private static Logger LOG = Logger.getLogger(RadosReader.class);
77 private ByteBuffer buffer;
78 private Pool pool;
79 private String object;
80 private long size;
81
82 public RadosReader(Pool pool, String object) throws Exception {
83 Stat stat;
84
85 this.pool = pool;
86 this.object = object;
87 buffer = ByteBuffer.allocate(102400/*TODO: NEIL WalrusQueryDispatcher.DATA_MESSAGE_SIZE */);
88 stat = pool.statObj(object);
89 if (stat == null) {
90 IOException ex = new IOException("Unable to stat object:" + object);
91 LOG.error(ex);
92 throw ex;
93 }
94 size = stat.getSize();
95 }
96
97 public int read(long offset) throws IOException {
98 byte[] buf;
99
100 buf = new byte[(int)(size - offset)];
101 buffer.clear();
102 int bytesRead = pool.readObj(object, offset, buf, buf.length);
103 buffer.put(buf, 0, bytesRead);
104 buffer.flip();
105 return bytesRead;
106 }
107
108 public void write(byte[] bytes) throws IOException {}
109
110 public ByteBuffer getBuffer() {
111 return buffer;
112 }
113
114 public void finish() {
115 pool.close();
116 }
117}
118
0119
=== added file 'clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosStorageManager.java'
--- clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosStorageManager.java 1970-01-01 00:00:00 +0000
+++ clc/modules/storage-rados/src/main/java/edu/ucsb/eucalyptus/storage/rados/RadosStorageManager.java 2010-11-15 08:38:14 +0000
@@ -0,0 +1,497 @@
1/*******************************************************************************
2 *Copyright (c) 2009 Eucalyptus Systems, Inc.
3 *
4 * This program is free software: you can redistribute it and/or modify
5 * it under the terms of the GNU General Public License as published by
6 * the Free Software Foundation, only version 3 of the License.
7 *
8 *
9 * This file is distributed in the hope that it will be useful, but WITHOUT
10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
12 * for more details.
13 *
14 * You should have received a copy of the GNU General Public License along
15 * with this program. If not, see <http://www.gnu.org/licenses/>.
16 *
17 * Please contact Eucalyptus Systems, Inc., 130 Castilian
18 * Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19 * if you need additional information or have any questions.
20 *
21 * This file may incorporate work covered under the following copyright and
22 * permission notice:
23 *
24 * Software License Agreement (BSD License)
25 *
26 * Copyright (c) 2008, Regents of the University of California
27 * All rights reserved.
28 *
29 * Redistribution and use of this software in source and binary forms, with
30 * or without modification, are permitted provided that the following
31 * conditions are met:
32 *
33 * Redistributions of source code must retain the above copyright notice,
34 * this list of conditions and the following disclaimer.
35 *
36 * Redistributions in binary form must reproduce the above copyright
37 * notice, this list of conditions and the following disclaimer in the
38 * documentation and/or other materials provided with the distribution.
39 *
40 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42 * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43 * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44 * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51 * THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52 * LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53 * SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54 * IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55 * BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56 * THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57 * OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58 * WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59 * ANY SUCH LICENSES OR RIGHTS.
60 *******************************************************************************/
61
62package edu.ucsb.eucalyptus.storage.rados;
63
64import com.eucalyptus.auth.util.Hashes;
65import com.eucalyptus.context.Contexts;
66import com.eucalyptus.http.MappingHttpResponse;
67import com.eucalyptus.util.EucalyptusCloudException;
68import com.eucalyptus.util.ExecutionException;
69import com.eucalyptus.util.WalrusProperties;
70import com.eucalyptus.ws.util.ChannelUtil;
71import com.eucalyptus.ws.util.WalrusBucketLogger;
72
73import edu.ucsb.eucalyptus.cloud.BucketLogData;
74import edu.ucsb.eucalyptus.cloud.ws.CompressedChunkedRadosObject;
75import edu.ucsb.eucalyptus.cloud.ws.ChunkedRadosObject;
76import edu.ucsb.eucalyptus.msgs.WalrusDataGetRequestType;
77import edu.ucsb.eucalyptus.storage.StorageManager;
78import edu.ucsb.eucalyptus.storage.StorageIO;
79import edu.ucsb.eucalyptus.util.StreamConsumer;
80import edu.ucsb.eucalyptus.util.SystemUtil;
81
82import edu.ucsb.eucalyptus.cloud.entities.WalrusInfo;
83
84import org.apache.log4j.Logger;
85import org.jboss.netty.channel.Channel;
86import org.jboss.netty.channel.ChannelFuture;
87import org.jboss.netty.channel.ChannelFutureListener;
88import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
89import org.jboss.netty.handler.codec.http.HttpHeaders;
90import org.jboss.netty.handler.stream.ChunkedInput;
91
92import java.io.*;
93import java.util.ArrayList;
94import java.util.List;
95
96import com.dokukino.rados4j.*;
97
98public class RadosStorageManager implements StorageManager {
99
100 public static final String lvmRootDirectory = "/dev";
101 private static boolean initialized = false;
102 private static String eucaHome = "/opt/eucalyptus";
103 public static final String EUCA_ROOT_WRAPPER = "/usr/lib/eucalyptus/euca_rootwrap";
104 public static final int MAX_LOOP_DEVICES = 256;
105 private static Logger LOG = Logger.getLogger(RadosStorageManager.class);
106 private Rados rados = new Rados();
107
108 public RadosStorageManager(WalrusInfo walrusInfo) {
109 rados.initialize(new String[0]);
110 }
111
112 public void setRootDirectory(String directory) {
113 }
114
115 public void checkPreconditions() throws EucalyptusCloudException {
116 try {
117 String eucaHomeDir = System.getProperty("euca.home");
118 if(eucaHomeDir == null) {
119 throw new EucalyptusCloudException("euca.home not set");
120 }
121 eucaHome = eucaHomeDir;
122 if(!new File(eucaHome + EUCA_ROOT_WRAPPER).exists()) {
123 throw new EucalyptusCloudException("root wrapper (euca_rootwrap) does not exist");
124 }
125 String returnValue = getLvmVersion();
126 if(returnValue.length() == 0) {
127 throw new EucalyptusCloudException("Is lvm installed?");
128 } else {
129 LOG.info(returnValue);
130 }
131 } catch(ExecutionException ex) {
132 String error = "Unable to run command: " + ex.getMessage();
133 LOG.error(error);
134 throw new EucalyptusCloudException(error);
135 }
136 }
137
138 public boolean bucketExists(String bucket) {
139 return rados.lookupPool(bucket);
140 }
141
142 public boolean objectExists(String bucket, String object) {
143 Pool pool = rados.openPool(bucket);
144 if (pool == null)
145 return false;
146 Stat stat = pool.statObj(object);
147 pool.close();
148 return (stat != null);
149 }
150
151 public void createBucket(String bucket) throws IOException {
152 int res = rados.createPool(bucket);
153 if (res != 0)
154 throw new IOException("Unable to create bucket: " + bucket);
155 }
156
157 public long getSize(String bucket, String object) {
158 Pool pool = rados.openPool(bucket);
159 if (pool == null)
160 return -1;
161 Stat stat = pool.statObj(object);
162 pool.close();
163 if (stat == null)
164 return -1;
165 return stat.getSize();
166 }
167
168 public void deleteBucket(String bucket) throws IOException {
169 Pool pool = rados.openPool(bucket);
170 if (pool == null)
171 throw new IOException("Unable to delete bucket: " + bucket);
172 int ret = pool.delete();
173 pool.close();
174 if (ret != 0)
175 throw new IOException("Unable to delete bucket: " + bucket);
176 }
177
178 public void createObject(String bucket, String object) throws IOException {
179 Pool pool = rados.openPool(bucket);
180 if (pool == null)
181 throw new IOException("Unable to open bucket: " + bucket);
182 int ret = pool.createObj(object, true);
183 pool.close();
184 if (ret != 0)
185 throw new IOException("Unable to create object: " + object);
186 }
187
188 public StorageIO prepareForRead(String bucket, String object) throws Exception {
189 Pool pool = rados.openPool(bucket);
190 if (pool == null)
191 throw new IOException("Unable to open bucket: " + bucket);
192 return new RadosReader(pool, object);
193 }
194
195 public StorageIO prepareForWrite(String bucket, String object) throws Exception {
196 Pool pool = rados.openPool(bucket);
197 if (pool == null)
198 throw new IOException("Unable to open bucket: " + bucket);
199 return new RadosWriter(pool, object);
200 }
201
202 public int readObject(String bucket, String object, byte[] bytes, long offset) throws IOException {
203 Pool pool = rados.openPool(bucket);
204 if (pool == null)
205 throw new IOException("Unable to open bucket: " + bucket);
206 int ret = pool.readObj(object, offset, bytes, 0);
207 pool.close();
208 return ret;
209 }
210
211 public int readObject(String path, byte[] bytes, long offset) throws IOException {
212 throw new IOException("Cannot use path on this implementation\n");
213 }
214
215 public void deleteObject(String bucket, String object) throws IOException {
216 Pool pool = rados.openPool(bucket);
217 if (pool == null)
218 throw new IOException("Unable to open bucket: " + bucket);
219 int ret = pool.removeObj(object);
220 pool.close();
221 if (ret < 0)
222 throw new IOException("Unable to delete object: " + object);
223 }
224
225 public void deleteAbsoluteObject(String object) throws IOException {
226 throw new IOException("Cannot use path on this implementation\n");
227 }
228
229 public void putObject(String bucket, String object, byte[] base64Data, boolean append) throws IOException {
230 Pool pool = rados.openPool(bucket);
231 if (pool == null)
232 throw new IOException("Unable to open bucket: " + bucket);
233 int ret;
234 if (!append)
235 ret = pool.writeObj(object, 0, base64Data, base64Data.length);
236 else {
237 Stat stat = pool.statObj(object);
238 if (stat == null) {
239 pool.close();
240 throw new IOException("Unable to stat object:" + bucket + "/" + object);
241 }
242 ret = pool.writeObj(object, stat.getSize(), base64Data, base64Data.length);
243 }
244 pool.close();
245 if (ret < 0)
246 throw new IOException("Unable to put object: " + bucket + "/" + object);
247 }
248
249 public void renameObject(String bucket, String oldName, String newName) throws IOException {
250 Pool pool;
251 int ret;
252
253 pool = rados.openPool(bucket);
254 if (pool == null)
255 throw new IOException("Unable to open bucket: " + bucket);
256 ret = pool.renameObj(oldName, newName);
257 if (ret < 0) {
258 pool.close();
259 throw new IOException("Unable to rename object: " + oldName);
260 }
261 pool.close();
262 }
263
264 public void copyObject(String sourceBucket, String sourceObject, String destinationBucket, String destinationObject) throws IOException {
265 Pool sourcePool, destinationPool;
266 int ret;
267
268 sourcePool = rados.openPool(sourceBucket);
269 if (sourcePool == null)
270 throw new IOException("Unable to open bucket: " + sourceBucket);
271 destinationPool = rados.openPool(destinationBucket);
272 if (destinationPool == null) {
273 sourcePool.close();
274 throw new IOException("Unable to open bucket: " + destinationBucket);
275 }
276 ret = sourcePool.copyObj(sourceObject, destinationPool, destinationObject);
277 if (ret < 0) {
278 sourcePool.close();
279 destinationPool.close();
280 throw new IOException("Unable to copy object: " + sourceObject);
281 }
282 sourcePool.close();
283 destinationPool.close();
284 }
285
286 public String getObjectPath(String bucket, String object) {
287 return null;
288 }
289
290 public long getObjectSize(String bucket, String object) {
291 Pool pool;
292 Stat stat;
293 long size;
294
295 pool = rados.openPool(bucket);
296 if (pool == null)
297 return -1;
298 stat = pool.statObj(object);
299 if (stat == null) {
300 pool.close();
301 return -1;
302 }
303 size = stat.getSize();
304 return size;
305 }
306
307 public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
308 try {
309 Channel channel = request.getChannel();
310 Pool pool = rados.openPool(bucketName);
311 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
312 if(etag != null)
313 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
314 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
315 if(contentDisposition != null)
316 httpResponse.addHeader("Content-Disposition", contentDisposition);
317 final ChunkedInput file;
318 isCompressed = isCompressed == null ? false : isCompressed;
319 if(isCompressed) {
320 file = new CompressedChunkedRadosObject(pool, objectName, size);
321 } else {
322 file = new ChunkedRadosObject(pool, objectName, 0, size, 5120000);
323 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
324 }
325 if(logData != null) {
326 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
327 logData.setBytesSent(size);
328 }
329 if(versionId != null) {
330 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
331 }
332 channel.write(httpResponse);
333 channel.write(file).addListener(new ChannelFutureListener( ) {
334 @Override public void operationComplete( ChannelFuture future ) throws Exception {
335 Contexts.clear(request.getCorrelationId());
336 file.close();
337 if(logData != null) {
338 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
339 WalrusBucketLogger.getInstance().addLogEntry(logData);
340 }
341 }
342 });
343 } catch(Exception ex) {
344 LOG.error(ex, ex);
345 }
346 }
347
348 public void sendObject(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, String bucketName, String objectName, long start, long end, long size, String etag, String lastModified, String contentType, String contentDisposition, Boolean isCompressed, String versionId, final BucketLogData logData) {
349 try {
350 Channel channel = request.getChannel();
351 Pool pool = rados.openPool(bucketName);
352 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
353 if(etag != null)
354 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
355 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
356 if(contentDisposition != null)
357 httpResponse.addHeader("Content-Disposition", contentDisposition);
358 final ChunkedInput file;
359 isCompressed = isCompressed == null ? false : isCompressed;
360 if(isCompressed) {
361 file = new CompressedChunkedRadosObject(pool, objectName, start, end, (int)Math.min((end - start), 8192));
362 } else {
363 file = new ChunkedRadosObject(pool, objectName, start, end, (int)Math.min((end - start), 8192));
364 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf((end - start)));
365 }
366 httpResponse.addHeader("Content-Range", start + "-" + end + "/" + size);
367 if(logData != null) {
368 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
369 logData.setBytesSent(size);
370 }
371 if(versionId != null) {
372 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
373 }
374 channel.write(httpResponse);
375 channel.write(file).addListener(new ChannelFutureListener( ) {
376 @Override public void operationComplete( ChannelFuture future ) throws Exception {
377 Contexts.clear(request.getCorrelationId());
378 file.close();
379 if(logData != null) {
380 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
381 WalrusBucketLogger.getInstance().addLogEntry(logData);
382 }
383 }
384 });
385 } catch(Exception ex) {
386 LOG.error(ex, ex);
387 }
388 }
389
390
391 public void sendHeaders(final WalrusDataGetRequestType request, DefaultHttpResponse httpResponse, Long size, String etag,
392 String lastModified, String contentType, String contentDisposition, String versionId, final BucketLogData logData) {
393 Channel channel = request.getChannel();
394 httpResponse.addHeader( HttpHeaders.Names.CONTENT_LENGTH, String.valueOf(size));
395 httpResponse.addHeader( HttpHeaders.Names.CONTENT_TYPE, contentType != null ? contentType : "binary/octet-stream" );
396 if(etag != null)
397 httpResponse.addHeader(HttpHeaders.Names.ETAG, etag);
398 httpResponse.addHeader(HttpHeaders.Names.LAST_MODIFIED, lastModified);
399 if(contentDisposition != null)
400 httpResponse.addHeader("Content-Disposition", contentDisposition);
401 if(versionId != null) {
402 httpResponse.addHeader(WalrusProperties.X_AMZ_VERSION_ID, versionId);
403 }
404 if(logData != null) {
405 logData.setTurnAroundTime(System.currentTimeMillis() - logData.getTurnAroundTime());
406 }
407 channel.write(httpResponse).addListener(new ChannelFutureListener( ) {
408 @Override public void operationComplete( ChannelFuture future ) throws Exception {
409 logData.setTotalTime(System.currentTimeMillis() - logData.getTotalTime());
410 WalrusBucketLogger.getInstance().addLogEntry(logData);
411 }
412 });
413 }
414
415 private String removeLoopback(String loDevName) throws ExecutionException {
416 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-d", loDevName});
417 }
418
419 private int losetup(String absoluteFileName, String loDevName) {
420 try
421 {
422 Runtime rt = Runtime.getRuntime();
423 Process proc = rt.exec(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", loDevName, absoluteFileName});
424 StreamConsumer error = new StreamConsumer(proc.getErrorStream());
425 StreamConsumer output = new StreamConsumer(proc.getInputStream());
426 error.start();
427 output.start();
428 int errorCode = proc.waitFor();
429 output.join();
430 LOG.info("losetup " + loDevName + " " + absoluteFileName);
431 LOG.info(output.getReturnValue());
432 LOG.info(error.getReturnValue());
433 return errorCode;
434 } catch (Throwable t) {
435 LOG.error(t);
436 }
437 return -1;
438 }
439
440 private String findFreeLoopback() throws ExecutionException {
441 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "losetup", "-f"}).replaceAll("\n", "");
442 }
443
444 private String removeLogicalVolume(String lvName) throws ExecutionException {
445 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "lvremove", "-f", lvName});
446 }
447
448 private String reduceVolumeGroup(String vgName, String pvName) throws ExecutionException {
449 return SystemUtil.run(new String[]{eucaHome + EUCA_ROOT_WRAPPER, "vgreduce", vgName, pvName});
450 }
451
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to status/vote changes: