OpenStack Compute (nova)

Merge lp:~yamahata/nova/boot-from-volume into lp:~hudson-openstack/nova/trunk

  1. boot-from-volume
  2. Merge into trunk
Proposed by Isaku Yamahata
Status: Work in progress
Proposed branch: lp:~yamahata/nova/boot-from-volume
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 605 lines (+246/-75)
8 files modified
nova/api/ec2/apirequest.py (+11/-5)
nova/api/ec2/cloud.py (+5/-1)
nova/api/openstack/servers.py (+50/-14)
nova/compute/api.py (+46/-20)
nova/compute/manager.py (+33/-3)
nova/virt/driver.py (+1/-1)
nova/virt/libvirt.xml.template (+33/-6)
nova/virt/libvirt_conn.py (+67/-25)
To merge this branch: bzr merge lp:~yamahata/nova/boot-from-volume
Reviewer Review Type Date Requested Status
Dan Prince (community) Needs Fixing
Vish Ishaya (community) Needs Fixing
Review via email: mp+58021@code.launchpad.net

Description of the change

This branch implements the first step for boot from volume.
With --block-device-mapping option for euca-run-instances, VM will boot with the specified volumes attached.
for example
 euca-run-instances ami-XXXX -k mykey -t m1.tiny -b/dev/vdb=vol-00000003::false

In fact since creating ec2 snapshot/volume from volume/snapshot isn't supported yet,
I used volume_id instead of snapshot_id.
This is first step to start discussion.

Code details.
- enhanced argument parser to interpret multi-dot separated argument
  like BlockDeviceMapping.1.DeviceName=snap-id
- pass down block device mapping id form nova-api to compute-api.
  compute-api changes volume status to in-use to tell the compute-manager which volume to attach
- compute-manager pass those infos to virt dirver and libvirt_conn driver interprets it.

TODO:
- error recovery
- ephemeral device/no device
- ami support which needs to change db schema
- suport on ec2 snapshot/clone
- native api in addition to ec2 api?

To post a comment you must log in.
lp:~yamahata/nova/boot-from-volume updated
994. By Yoshiaki Tamura

Fix parameter mismatch calling to_xml() from spawn() in libvirt_conn.py

995. By termie

The change to utils.execute's call style missed this call somehow, this should get libvirt snapshots working again.

Revision history for this message
Masanori Itoh (itohm) wrote :

Hi Isaku,

Great contribution!
Actually we were also discussing developing this feature internally.

BTW, this is a POC code of a not-approved-yet feature, isn't it?
New feature will not be merged into trunk till the blueprint is approved and discussed at Design Summit in the OpenStack world.
I guess that Adam Johnson of Midokura will have a session on the blueprint of this feature
at the upcoming Diablo Design Summit. So, what about holding the branch somewhere outside
the trunk for a while?

Thanks,
Masanori

Reply
Revision history for this message
Jay Pipes (jaypipes) wrote :

Hi! Looks like a great contribution indeed, and I agree with Masanori's points about discussing at the summit. I wanted to add one more note, though, that we would want to see some unit tests that stress the new code paths. Let us know if you need assistance creating those tests. :)

Cheers!
jay

Reply
lp:~yamahata/nova/boot-from-volume updated
996. By Vish Ishaya

Fixes nova-manage image convert when the source directory is the same one that local image service uses.

997. By Jason Kölker

Change '== None' to 'is None'

998. By Mark Washenberger

Support admin password when specified in server create requests.

999. By Eldar Nugaev

Fix loggin in creation server in OpenStack API 1.0

1000. By Jason Kölker

use 'is not None' instead of '!= None'

1001. By Jason Kölker

Remove zope.interface from the requires file since it is not used anywhere.

1002. By Jason Kölker

pep8 fixes

1003. By Anne Gentle

Adding projectname username to the nova-manage project commands to fix a doc bug, plus some edits and elimination of a few doc todos.

Revision history for this message
Isaku Yamahata (yamahata) wrote :

On Mon, Apr 18, 2011 at 03:48:14PM -0000, Masanori Itoh wrote:
> Hi Isaku,
>
> Great contribution!
> Actually we were also discussing developing this feature internally.
>
> BTW, this is a POC code of a not-approved-yet feature, isn't it?
> New feature will not be merged into trunk till the blueprint is approved and discussed at Design Summit in the OpenStack world.

Yes. I just didn't know what I should do with the new code.
I'm learning new rules as a new comer... (sometimes by making mistakes)

> I guess that Adam Johnson of Midokura will have a session on the blueprint of this feature
> at the upcoming Diablo Design Summit. So, what about holding the branch somewhere outside
> the trunk for a while?

Great idea. I'm looking forward to it.

>
> Thanks,
> Masanori
>
>
> --
> https://code.launchpad.net/~yamahata/nova/boot-from-volume/+merge/58021
> You are the owner of lp:~yamahata/nova/boot-from-volume.
>

--
yamahata

Reply
lp:~yamahata/nova/boot-from-volume updated
1004. By Dan Prince

Implement quotas for the new v1.1 server metadata controller.

Created a new _check_metadata_properties_quota method in the compute API that is used when creating instances and when updating server metadata. In doing so I modified the compute API so that metadata is a dict (not an array) to ensure we are using unique key values for metadata (which is implied by the API specs) and makes more sense with JSON request formats anyway.

Additionally this branch enables and fixes the integration test to create servers with metadata.

1005. By Josh Kearney

Round 1 of pylint cleanup.

1006. By termie

attempts to make the docstring rules clearer

1007. By termie

Docstring cleanup and formatting. Minor style fixes as well.

1008. By Naveed Massjouni

Added an option to run_tests.sh so you can run just pep8. So now you can:
    ./run_tests.sh --just-pep8
or
    ./run_tests.sh -p

1009. By Josh Kearney

Another small round of pylint clean-up.

1013. By Yoshiaki Tamura

If volumes exist in instance, get pathes to the volumes and convert
them to the xml format to let libvirt to see them upon booting.

1014. By Yoshiaki Tamura

Extend create() to accept volume, and update DB to reserve the volume
before passing it to the manager.

1015. By Yoshiaki Tamura

Add an parameter to specify volume upon creating instances.

Revision history for this message
Vish Ishaya (vishvananda) wrote :

This is looking pretty good.

I like the direction that it is going and I think merging step-by-step is the right way. We definitely need some tests to cover this feature. Some sort of unit tests are a must. It would also be great to get a smoketest for this feature so that we can be sure that it is working against a real deployment.

review: Needs Fixing
Reply
Revision history for this message
Dan Prince (dan-prince) wrote :

This branch no longer merges cleanly w/ trunk.

Needs fixing.

review: Needs Fixing
Reply

Unmerged revisions

1015. By Yoshiaki Tamura

Add an parameter to specify volume upon creating instances.

1014. By Yoshiaki Tamura

Extend create() to accept volume, and update DB to reserve the volume
before passing it to the manager.

1013. By Yoshiaki Tamura

If volumes exist in instance, get pathes to the volumes and convert
them to the xml format to let libvirt to see them upon booting.

1012. By Isaku Yamahata

ebs boot: compute node(kvm and libvirt) support for ebs boot

this patch teaches kvm and libvirt compute node ebs boot.

1011. By Isaku Yamahata

ebs boot: add parse for ebs boot argument

This patch adds the parser of ebs boot argument and stores those
infos into Volume table for compute node.

1010. By Isaku Yamahata

api/ec2/api: teach multi dot-separated argument

nova.api.ec2.apirequest.APIRequest knows only single dot-separated
arguments.
EBS boot uses multi dot-separeted arguments like
BlockDeviceMapping.1.DeviceName=snap-id
This patch teaches the parser those argument as the preparetion for ebs boot
support.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'nova/api/ec2/apirequest.py'
--- nova/api/ec2/apirequest.py 2011-04-18 20:53:09 +0000
+++ nova/api/ec2/apirequest.py 2011-04-21 04:09:46 +0000
@@ -133,11 +133,17 @@
133 # NOTE(vish): Automatically convert strings back133 # NOTE(vish): Automatically convert strings back
134 # into their respective values134 # into their respective values
135 value = _try_convert(value)135 value = _try_convert(value)
136 if len(parts) > 1:136
137 d = args.get(key, {})137 if len(parts) > 1:
138 d[parts[1]] = value138 d = args.get(key, {})
139 value = d139 args[key] = d
140 args[key] = value140 for k in parts[1:-1]:
141 v = d.get(k, {})
142 d[k] = v
143 d = v
144 d[parts[-1]] = value
145 else:
146 args[key] = value
141147
142 for key in args.keys():148 for key in args.keys():
143 # NOTE(vish): Turn numeric dict keys into lists149 # NOTE(vish): Turn numeric dict keys into lists
144150
=== modified file 'nova/api/ec2/cloud.py'
--- nova/api/ec2/cloud.py 2011-04-20 23:21:37 +0000
+++ nova/api/ec2/cloud.py 2011-04-21 04:09:46 +0000
@@ -822,6 +822,9 @@
822 if kwargs.get('ramdisk_id'):822 if kwargs.get('ramdisk_id'):
823 ramdisk = self._get_image(context, kwargs['ramdisk_id'])823 ramdisk = self._get_image(context, kwargs['ramdisk_id'])
824 kwargs['ramdisk_id'] = ramdisk['id']824 kwargs['ramdisk_id'] = ramdisk['id']
825 for bdm in kwargs.get('block_device_mapping', []):
826 volume_id = ec2utils.ec2_id_to_id(bdm['Ebs']['SnapshotId'])
827 bdm['Ebs']['SnapshotId'] = volume_id
825 instances = self.compute_api.create(context,828 instances = self.compute_api.create(context,
826 instance_type=instance_types.get_instance_type_by_name(829 instance_type=instance_types.get_instance_type_by_name(
827 kwargs.get('instance_type', None)),830 kwargs.get('instance_type', None)),
@@ -836,7 +839,8 @@
836 user_data=kwargs.get('user_data'),839 user_data=kwargs.get('user_data'),
837 security_group=kwargs.get('security_group'),840 security_group=kwargs.get('security_group'),
838 availability_zone=kwargs.get('placement', {}).get(841 availability_zone=kwargs.get('placement', {}).get(
839 'AvailabilityZone'))842 'AvailabilityZone'),
843 block_device_mapping=kwargs.get('block_device_mapping', {}))
840 return self._format_run_instances(context,844 return self._format_run_instances(context,
841 instances[0]['reservation_id'])845 instances[0]['reservation_id'])
842846
843847
=== modified file 'nova/api/openstack/servers.py'
--- nova/api/openstack/servers.py 2011-04-19 17:46:43 +0000
+++ nova/api/openstack/servers.py 2011-04-21 04:09:46 +0000
@@ -52,7 +52,7 @@
52 "attributes": {52 "attributes": {
53 "server": ["id", "imageId", "name", "flavorId", "hostId",53 "server": ["id", "imageId", "name", "flavorId", "hostId",
54 "status", "progress", "adminPass", "flavorRef",54 "status", "progress", "adminPass", "flavorRef",
55 "imageRef"],55 "imageRef", "volume"],
56 "link": ["rel", "type", "href"],56 "link": ["rel", "type", "href"],
57 },57 },
58 "dict_collections": {58 "dict_collections": {
@@ -129,15 +129,27 @@
129 key_data = key_pair['public_key']129 key_data = key_pair['public_key']
130130
131 requested_image_id = self._image_id_from_req_data(env)131 requested_image_id = self._image_id_from_req_data(env)
132 try:132
133 image_id = common.get_image_id_from_image_hash(self._image_service,133 volume = self._volume_from_req_data(env)
134 context, requested_image_id)134
135 except:135 # image_id or volume must be set to create
136 msg = _("Can not find requested image")136 if not requested_image_id and not volume:
137 return faults.Fault(exc.HTTPBadRequest(msg))137 msg = _("imageRef or volume must be specified")
138138 return exc.HTTPBadRequest(msg)
139 kernel_id, ramdisk_id = self._get_kernel_ramdisk_from_image(139
140 req, image_id)140 image_id = None
141 kernel_id = None
142 ramdisk_id = None
143 if requested_image_id:
144 try:
145 image_id = common.get_image_id_from_image_hash(
146 self._image_service, context, requested_image_id)
147 except:
148 msg = _("Can not find requested image")
149 return faults.Fault(exc.HTTPBadRequest(msg))
150
151 kernel_id, ramdisk_id = self._get_kernel_ramdisk_from_image(
152 req, image_id)
141153
142 personality = env['server'].get('personality')154 personality = env['server'].get('personality')
143 injected_files = []155 injected_files = []
@@ -154,6 +166,12 @@
154 self._validate_server_name(name)166 self._validate_server_name(name)
155 name = name.strip()167 name = name.strip()
156168
169 # FIXME(yamahata): convert volume into EC2 block device mapping
170 block_device_mapping=[]
171 if volume:
172 block_device_mapping.append({'DeviceName': volume['mountpoint'],
173 'Ebs': {'SnapshotId': volume['id']}})
174
157 try:175 try:
158 inst_type = \176 inst_type = \
159 instance_types.get_instance_type_by_flavor_id(flavor_id)177 instance_types.get_instance_type_by_flavor_id(flavor_id)
@@ -163,12 +181,14 @@
163 image_id,181 image_id,
164 kernel_id=kernel_id,182 kernel_id=kernel_id,
165 ramdisk_id=ramdisk_id,183 ramdisk_id=ramdisk_id,
184 volume=volume,
166 display_name=name,185 display_name=name,
167 display_description=name,186 display_description=name,
168 key_name=key_name,187 key_name=key_name,
169 key_data=key_data,188 key_data=key_data,
170 metadata=env['server'].get('metadata', {}),189 metadata=env['server'].get('metadata', {}),
171 injected_files=injected_files)190 injected_files=injected_files,
191 block_device_mapping=block_device_mapping)
172 except quota.QuotaError as error:192 except quota.QuotaError as error:
173 self._handle_quota_error(error)193 self._handle_quota_error(error)
174194
@@ -585,7 +605,10 @@
585605
586class ControllerV10(Controller):606class ControllerV10(Controller):
587 def _image_id_from_req_data(self, data):607 def _image_id_from_req_data(self, data):
588 return data['server']['imageId']608 try:
609 return data['server']['imageId']
610 except KeyError:
611 return
589612
590 def _flavor_id_from_req_data(self, data):613 def _flavor_id_from_req_data(self, data):
591 return data['server']['flavorId']614 return data['server']['flavorId']
@@ -612,13 +635,26 @@
612635
613class ControllerV11(Controller):636class ControllerV11(Controller):
614 def _image_id_from_req_data(self, data):637 def _image_id_from_req_data(self, data):
615 href = data['server']['imageRef']638 try:
616 return common.get_id_from_href(href)639 href = data['server']['imageRef']
640 return common.get_id_from_href(href)
641 except KeyError:
642 return
617643
618 def _flavor_id_from_req_data(self, data):644 def _flavor_id_from_req_data(self, data):
619 href = data['server']['flavorRef']645 href = data['server']['flavorRef']
620 return common.get_id_from_href(href)646 return common.get_id_from_href(href)
621647
648 def _volume_from_req_data(self, data):
649 try:
650 volume = {
651 'id': common.get_id_from_href(data['server']['volume']['id']),
652 'mountpoint': data['server']['volume']['mountpoint']
653 }
654 return volume
655 except KeyError:
656 return
657
622 def _get_view_builder(self, req):658 def _get_view_builder(self, req):
623 base_url = req.application_url659 base_url = req.application_url
624 flavor_builder = nova.api.openstack.views.flavors.ViewBuilderV11(660 flavor_builder = nova.api.openstack.views.flavors.ViewBuilderV11(
625661
=== modified file 'nova/compute/api.py'
--- nova/compute/api.py 2011-04-20 19:11:14 +0000
+++ nova/compute/api.py 2011-04-21 04:09:46 +0000
@@ -124,13 +124,37 @@
124 LOG.warn(msg)124 LOG.warn(msg)
125 raise quota.QuotaError(msg, "MetadataLimitExceeded")125 raise quota.QuotaError(msg, "MetadataLimitExceeded")
126126
127 def _get_kernel_and_ramdisk(self, context, image_id,
128 kernel_id=None, ramdisk_id=None):
129 if not image_id:
130 return (None, None, None)
131
132 image = self.image_service.show(context, image_id)
133
134 os_type = None
135 if 'properties' in image and 'os_type' in image['properties']:
136 os_type = image['properties']['os_type']
137
138 if kernel_id is None:
139 kernel_id = image['properties'].get('kernel_id', None)
140 if ramdisk_id is None:
141 ramdisk_id = image['properties'].get('ramdisk_id', None)
142 # FIXME(sirp): is there a way we can remove null_kernel?
143 # No kernel and ramdisk for raw images
144 if kernel_id == str(FLAGS.null_kernel):
145 kernel_id = None
146 ramdisk_id = None
147 LOG.debug(_("Creating a raw instance"))
148
149 return (kernel_id, ramdisk_id, os_type)
150
127 def create(self, context, instance_type,151 def create(self, context, instance_type,
128 image_id, kernel_id=None, ramdisk_id=None,152 image_id, kernel_id=None, ramdisk_id=None,
129 min_count=1, max_count=1,153 min_count=1, max_count=1,
130 display_name='', display_description='',154 display_name='', display_description='',
131 key_name=None, key_data=None, security_group='default',155 key_name=None, key_data=None, security_group='default',
132 availability_zone=None, user_data=None, metadata={},156 availability_zone=None, user_data=None, metadata={},
133 injected_files=None):157 injected_files=None, block_device_mapping=[]):
134 """Create the number and type of instances requested.158 """Create the number and type of instances requested.
135159
136 Verifies that quota and other arguments are valid.160 Verifies that quota and other arguments are valid.
@@ -152,22 +176,10 @@
152 self._check_metadata_properties_quota(context, metadata)176 self._check_metadata_properties_quota(context, metadata)
153 self._check_injected_file_quota(context, injected_files)177 self._check_injected_file_quota(context, injected_files)
154178
155 image = self.image_service.show(context, image_id)179 (kernel_id, ramdisk_id, os_type) = \
156180 self._get_kernel_and_ramdisk(context, image_id,
157 os_type = None181 kernel_id, ramdisk_id)
158 if 'properties' in image and 'os_type' in image['properties']:182
159 os_type = image['properties']['os_type']
160
161 if kernel_id is None:
162 kernel_id = image['properties'].get('kernel_id', None)
163 if ramdisk_id is None:
164 ramdisk_id = image['properties'].get('ramdisk_id', None)
165 # FIXME(sirp): is there a way we can remove null_kernel?
166 # No kernel and ramdisk for raw images
167 if kernel_id == str(FLAGS.null_kernel):
168 kernel_id = None
169 ramdisk_id = None
170 LOG.debug(_("Creating a raw instance"))
171 # Make sure we have access to kernel and ramdisk (if not raw)183 # Make sure we have access to kernel and ramdisk (if not raw)
172 logging.debug("Using Kernel=%s, Ramdisk=%s" %184 logging.debug("Using Kernel=%s, Ramdisk=%s" %
173 (kernel_id, ramdisk_id))185 (kernel_id, ramdisk_id))
@@ -215,7 +227,7 @@
215 'locked': False,227 'locked': False,
216 'metadata': metadata,228 'metadata': metadata,
217 'availability_zone': availability_zone,229 'availability_zone': availability_zone,
218 'os_type': os_type}230 'os_type': os_type or ''}
219 elevated = context.elevated()231 elevated = context.elevated()
220 instances = []232 instances = []
221 LOG.debug(_("Going to run %s instances..."), num_instances)233 LOG.debug(_("Going to run %s instances..."), num_instances)
@@ -234,6 +246,17 @@
234 instance_id,246 instance_id,
235 security_group_id)247 security_group_id)
236248
249 # tell vm driver to attach volume at boot time by
250 # updating the volume table if volume_id is attachable
251 # TODO(yoshi) Change mount point to be bootable
252 # TODO(yamahata): eliminate EC2 dependency
253 for bdm in block_device_mapping:
254 device = bdm['DeviceName']
255 volume_id = bdm['Ebs']['SnapshotId']
256 self._check_volume_device(context, volume_id, device)
257 self.db.volume_attached(context, volume_id, instance_id,
258 device)
259
237 # Set sane defaults if not specified260 # Set sane defaults if not specified
238 updates = dict(hostname=self.hostname_factory(instance_id))261 updates = dict(hostname=self.hostname_factory(instance_id))
239 if (not hasattr(instance, 'display_name') or262 if (not hasattr(instance, 'display_name') or
@@ -672,12 +695,15 @@
672 """Inject network info for the instance."""695 """Inject network info for the instance."""
673 self._cast_compute_message('inject_network_info', context, instance_id)696 self._cast_compute_message('inject_network_info', context, instance_id)
674697
675 def attach_volume(self, context, instance_id, volume_id, device):698 def _check_volume_device(self, context, volume_id, device):
676 """Attach an existing volume to an existing instance."""
677 if not re.match("^/dev/[a-z]d[a-z]+$", device):699 if not re.match("^/dev/[a-z]d[a-z]+$", device):
678 raise exception.ApiError(_("Invalid device specified: %s. "700 raise exception.ApiError(_("Invalid device specified: %s. "
679 "Example device: /dev/vdb") % device)701 "Example device: /dev/vdb") % device)
680 self.volume_api.check_attach(context, volume_id=volume_id)702 self.volume_api.check_attach(context, volume_id=volume_id)
703
704 def attach_volume(self, context, instance_id, volume_id, device):
705 """Attach an existing volume to an existing instance."""
706 self._check_volume_device(context, volume_id, device)
681 instance = self.get(context, instance_id)707 instance = self.get(context, instance_id)
682 host = instance['host']708 host = instance['host']
683 rpc.cast(context,709 rpc.cast(context,
684710
=== modified file 'nova/compute/manager.py'
--- nova/compute/manager.py 2011-04-19 18:29:16 +0000
+++ nova/compute/manager.py 2011-04-21 04:09:46 +0000
@@ -224,6 +224,29 @@
224 self.network_manager.setup_compute_network(context,224 self.network_manager.setup_compute_network(context,
225 instance_id)225 instance_id)
226226
227 # setup volume:
228 # Now here ebs is specified by volume id.
229 # TODO:
230 # When snapshot is supported, create volume from snapshot here.
231 block_device_mapping = []
232 for volume in instance_ref['volumes']:
233 dev_path = self.volume_manager.setup_compute_volume(context,
234 volume['id'])
235 if dev_path.startswith('/dev/'):
236 info = {'type': 'block'
237 'device_path': dev_path,
238 'mount_device': volume['mountpoint']}
239 elif ':' in dev_path:
240 (protocol, name) = dev_path.split(':')
241 info = {'type': 'network',
242 'protocol': protocol,
243 'name': name,
244 'mount_device': volume['mountpoint']}
245 block_device_mapping.append(info)
246
247 # FIXME:XXX determine when to cdboot.
248 #boot = 'cdrom' if instance_ref['image_id'] and volume_info else 'hd'
249
227 # TODO(vish) check to make sure the availability zone matches250 # TODO(vish) check to make sure the availability zone matches
228 self.db.instance_set_state(context,251 self.db.instance_set_state(context,
229 instance_id,252 instance_id,
@@ -231,7 +254,9 @@
231 'spawning')254 'spawning')
232255
233 try:256 try:
234 self.driver.spawn(instance_ref)257 self.driver.spawn(instance_ref,
258 block_device_mapping=block_device_mapping,
259 boot=boot)
235 now = datetime.datetime.utcnow()260 now = datetime.datetime.utcnow()
236 self.db.instance_update(context,261 self.db.instance_update(context,
237 instance_id,262 instance_id,
@@ -243,6 +268,10 @@
243 self.db.instance_set_state(context,268 self.db.instance_set_state(context,
244 instance_id,269 instance_id,
245 power_state.SHUTDOWN)270 power_state.SHUTDOWN)
271 for volume in instance_ref['volumes']:
272 self.volume_manager.remove_compute_volume(context,
273 volume['id'])
274 self.db.volume_detached(context, volume['id'])
246275
247 self._update_state(context, instance_id)276 self._update_state(context, instance_id)
248277
@@ -963,8 +992,9 @@
963992
964 # Detaching volumes.993 # Detaching volumes.
965 try:994 try:
966 for vol in self.db.volume_get_all_by_instance(ctxt, instance_id):995 for volume in self.db.volume_get_all_by_instance(ctxt,
967 self.volume_manager.remove_compute_volume(ctxt, vol['id'])996 instance_id):
997 self.volume_manager.remove_compute_volume(ctxt, volume['id'])
968 except exception.NotFound:998 except exception.NotFound:
969 pass999 pass
9701000
9711001
=== modified file 'nova/virt/driver.py'
--- nova/virt/driver.py 2011-03-30 00:35:24 +0000
+++ nova/virt/driver.py 2011-04-21 04:09:46 +0000
@@ -61,7 +61,7 @@
61 """Return a list of InstanceInfo for all registered VMs"""61 """Return a list of InstanceInfo for all registered VMs"""
62 raise NotImplementedError()62 raise NotImplementedError()
6363
64 def spawn(self, instance, network_info=None):64 def spawn(self, instance, network_info=None, block_device_mapping=[]):
65 """Launch a VM for the specified instance"""65 """Launch a VM for the specified instance"""
66 raise NotImplementedError()66 raise NotImplementedError()
6767
6868
=== modified file 'nova/virt/libvirt.xml.template'
--- nova/virt/libvirt.xml.template 2011-03-30 05:59:13 +0000
+++ nova/virt/libvirt.xml.template 2011-04-21 04:09:46 +0000
@@ -39,7 +39,7 @@
39 <initrd>${ramdisk}</initrd>39 <initrd>${ramdisk}</initrd>
40 #end if40 #end if
41 #else41 #else
42 <boot dev="hd" />42 <boot dev='${boot}' />
43 #end if43 #end if
44 #end if44 #end if
45#end if45#end if
@@ -67,11 +67,24 @@
67 <target dev='${disk_prefix}b' bus='${disk_bus}'/>67 <target dev='${disk_prefix}b' bus='${disk_bus}'/>
68 </disk>68 </disk>
69 #else69 #else
70 <disk type='file'>70
71 <driver type='${driver_type}'/>71 ## FIXME: allow no device
72 <source file='${basepath}/disk'/>72 #if $getVar('disk', False)
73 <target dev='${disk_prefix}a' bus='${disk_bus}'/>73 #if $boot == 'cdrom'
74 </disk>74 <disk type='file' device='cdrom'>
75 <driver type='${driver_type}'/>
76 <source file='${basepath}/disk'/>
77 <target dev='hdc' bus='ide'/>
78 </disk>
79 #else if not ($getVar('ebs_root', False))
80 <disk type='file' device='disk'>
81 <driver type='${driver_type}'/>
82 <source file='${basepath}/disk'/>
83 <target dev='${disk_prefix}a' bus='${disk_bus}'/>
84 </disk>
85 #end if
86 #end if
87
75 #if $getVar('local', False)88 #if $getVar('local', False)
76 <disk type='file'>89 <disk type='file'>
77 <driver type='${driver_type}'/>90 <driver type='${driver_type}'/>
@@ -79,6 +92,20 @@
79 <target dev='${disk_prefix}b' bus='${disk_bus}'/>92 <target dev='${disk_prefix}b' bus='${disk_bus}'/>
80 </disk>93 </disk>
81 #end if94 #end if
95 #for $volume in $volumes
96 #if varExists('volume.type')
97 #set $volume.type = 'block'
98 #end if
99 <disk type='${volume.type}'>
100 <driver type='raw'/>
101 #if $volume.type == 'network'
102 <source protocol='${volume.protocol}' name='${volume.name}'/>
103 #else
104 <source dev='${volume.device_path}'/>
105 #end if
106 <target dev='${volume.mount_device}' bus='${disk_bus}'/>
107 </disk>
108 #end for
82 #end if109 #end if
83#end if110#end if
84111
85112
=== modified file 'nova/virt/libvirt_conn.py'
--- nova/virt/libvirt_conn.py 2011-04-18 23:40:03 +0000
+++ nova/virt/libvirt_conn.py 2011-04-21 04:09:46 +0000
@@ -39,6 +39,7 @@
39import multiprocessing39import multiprocessing
40import os40import os
41import random41import random
42import re
42import shutil43import shutil
43import subprocess44import subprocess
44import sys45import sys
@@ -202,6 +203,10 @@
202 network_info.append((network, mapping))203 network_info.append((network, mapping))
203 return network_info204 return network_info
204205
206# TODO: clean up open coding like
207# volume['mountpoint'] = volume['mountpoint'].rpartition("/")[2]
208def _strip_dev(mount_path):
209 return re.sub(r'^/dev/', '', mount_path)
205210
206class LibvirtConnection(driver.ComputeDriver):211class LibvirtConnection(driver.ComputeDriver):
207212
@@ -608,15 +613,19 @@
608 # NOTE(ilyaalekseyev): Implementation like in multinics613 # NOTE(ilyaalekseyev): Implementation like in multinics
609 # for xenapi(tr3buchet)614 # for xenapi(tr3buchet)
610 @exception.wrap_exception615 @exception.wrap_exception
611 def spawn(self, instance, network_info=None):616 def spawn(self, instance, network_info=None,
612 xml = self.to_xml(instance, False, network_info)617 block_device_mapping=[], boot=None):
618 xml = self.to_xml(instance, False, network_info=network_info,
619 block_device_mapping=block_device_mapping,
620 boot=boot)
613 db.instance_set_state(context.get_admin_context(),621 db.instance_set_state(context.get_admin_context(),
614 instance['id'],622 instance['id'],
615 power_state.NOSTATE,623 power_state.NOSTATE,
616 'launching')624 'launching')
617 self.firewall_driver.setup_basic_filtering(instance, network_info)625 self.firewall_driver.setup_basic_filtering(instance, network_info)
618 self.firewall_driver.prepare_instance_filter(instance, network_info)626 self.firewall_driver.prepare_instance_filter(instance, network_info)
619 self._create_image(instance, xml, network_info)627 self._create_image(instance, xml, network_info=network_info,
628 block_device_mapping=block_device_mapping)
620 domain = self._create_new_domain(xml)629 domain = self._create_new_domain(xml)
621 LOG.debug(_("instance %s: is running"), instance['name'])630 LOG.debug(_("instance %s: is running"), instance['name'])
622 self.firewall_driver.apply_instance_filter(instance)631 self.firewall_driver.apply_instance_filter(instance)
@@ -797,7 +806,7 @@
797 # TODO(vish): should we format disk by default?806 # TODO(vish): should we format disk by default?
798807
799 def _create_image(self, inst, libvirt_xml, suffix='', disk_images=None,808 def _create_image(self, inst, libvirt_xml, suffix='', disk_images=None,
800 network_info=None):809 network_info=None, block_device_mapping=[]):
801 if not network_info:810 if not network_info:
802 network_info = _get_network_info(inst)811 network_info = _get_network_info(inst)
803812
@@ -851,25 +860,28 @@
851 user=user,860 user=user,
852 project=project)861 project=project)
853862
854 root_fname = '%08x' % int(disk_images['image_id'])
855 size = FLAGS.minimum_root_size
856
857 inst_type_id = inst['instance_type_id']863 inst_type_id = inst['instance_type_id']
858 inst_type = instance_types.get_instance_type(inst_type_id)864 inst_type = instance_types.get_instance_type(inst_type_id)
859 if inst_type['name'] == 'm1.tiny' or suffix == '.rescue':865
860 size = None866 if disk_image['image_id'] and
861 root_fname += "_sm"867 (not self._volume_in_mapping(self.root_mount_device,
862868 block_device_mapping)):
863 self._cache_image(fn=self._fetch_image,869 root_fname = '%08x' % int(disk_images['image_id'])
864 target=basepath('disk'),870 size = FLAGS.minimum_root_size
865 fname=root_fname,871 if inst_type['name'] == 'm1.tiny' or suffix == '.rescue':
866 cow=FLAGS.use_cow_images,872 size = None
867 image_id=disk_images['image_id'],873 root_fname += "_sm"
868 user=user,874 self._cache_image(fn=self._fetch_image,
869 project=project,875 target=basepath('disk'),
870 size=size)876 fname=root_fname,
871877 cow=FLAGS.use_cow_images,
872 if inst_type['local_gb']:878 image_id=disk_images['image_id'],
879 user=user,
880 project=project,
881 size=size)
882
883 if inst_type['local_gb'] and not self._volume_in_mapping(
884 self.local_mount_device, block_device_mapping):
873 self._cache_image(fn=self._create_local,885 self._cache_image(fn=self._create_local,
874 target=basepath('disk.local'),886 target=basepath('disk.local'),
875 fname="local_%s" % inst_type['local_gb'],887 fname="local_%s" % inst_type['local_gb'],
@@ -994,7 +1006,18 @@
9941006
995 return result1007 return result
9961008
997 def to_xml(self, instance, rescue=False, network_info=None):1009 root_mount_device = 'vda' # FIXME for now. it's hard coded.
1010 local_mount_device = 'vdb' # FIXME for now. it's hard coded.
1011 def _volume_in_mapping(self, mount_device, block_device_mapping):
1012 mount_device_ = _strip_dev(mount_device)
1013 for vol in block_device_mapping:
1014 vol_mount_device = _strip_dev(vol['mount_device'])
1015 if vol_mount_device == mount_device_:
1016 return True
1017 return False
1018
1019 def to_xml(self, instance, rescue=False,
1020 network_info=None, block_device_mapping=[], boot='hd'):
998 # TODO(termie): cache?1021 # TODO(termie): cache?
999 LOG.debug(_('instance %s: starting toXML method'), instance['name'])1022 LOG.debug(_('instance %s: starting toXML method'), instance['name'])
10001023
@@ -1007,6 +1030,7 @@
1007 for (network, mapping) in network_info:1030 for (network, mapping) in network_info:
1008 nics.append(self._get_nic_for_xml(network,1031 nics.append(self._get_nic_for_xml(network,
1009 mapping))1032 mapping))
1033
1010 # FIXME(vish): stick this in db1034 # FIXME(vish): stick this in db
1011 inst_type_id = instance['instance_type_id']1035 inst_type_id = instance['instance_type_id']
1012 inst_type = instance_types.get_instance_type(inst_type_id)1036 inst_type = instance_types.get_instance_type(inst_type_id)
@@ -1016,6 +1040,20 @@
1016 else:1040 else:
1017 driver_type = 'raw'1041 driver_type = 'raw'
10181042
1043 #for volume in block_device_mapping:
1044 # volume['mountpoint'] = volume['mountpoint'].rpartition("/")[2]
1045 for vol in block_device_mapping:
1046 vol['mount_device'] = _strip_dev(vol['mount_device'])
1047
1048
1049 ebs_root = self._volume_in_mapping(self.root_mount_device,
1050 block_device_mapping)
1051 if self._volume_in_mapping(self.local_mount_device,
1052 block_device_mapping):
1053 local_gb = False
1054 else:
1055 local_gb = inst_type['local_gb']
1056
1019 xml_info = {'type': FLAGS.libvirt_type,1057 xml_info = {'type': FLAGS.libvirt_type,
1020 'name': instance['name'],1058 'name': instance['name'],
1021 'basepath': os.path.join(FLAGS.instances_path,1059 'basepath': os.path.join(FLAGS.instances_path,
@@ -1023,9 +1061,12 @@
1023 'memory_kb': inst_type['memory_mb'] * 1024,1061 'memory_kb': inst_type['memory_mb'] * 1024,
1024 'vcpus': inst_type['vcpus'],1062 'vcpus': inst_type['vcpus'],
1025 'rescue': rescue,1063 'rescue': rescue,
1026 'local': inst_type['local_gb'],1064 'local': local_gb,
1065 'boot': boot,
1027 'driver_type': driver_type,1066 'driver_type': driver_type,
1028 'nics': nics}1067 'nics': nics,
1068 'ebs_root': ebs_root,
1069 'volumes': block_device_mapping}
10291070
1030 if FLAGS.vnc_enabled:1071 if FLAGS.vnc_enabled:
1031 if FLAGS.libvirt_type != 'lxc':1072 if FLAGS.libvirt_type != 'lxc':
@@ -1037,7 +1078,8 @@
1037 if instance['ramdisk_id']:1078 if instance['ramdisk_id']:
1038 xml_info['ramdisk'] = xml_info['basepath'] + "/ramdisk"1079 xml_info['ramdisk'] = xml_info['basepath'] + "/ramdisk"
10391080
1040 xml_info['disk'] = xml_info['basepath'] + "/disk"1081 if instance['image_id']:
1082 xml_info['disk'] = xml_info['basepath'] + "/disk"
10411083
1042 xml = str(Template(self.libvirt_xml, searchList=[xml_info]))1084 xml = str(Template(self.libvirt_xml, searchList=[xml_info]))
1043 LOG.debug(_('instance %s: finished toXML method'),1085 LOG.debug(_('instance %s: finished toXML method'),