Create per-partition images
LAVA has a need to access individual partitions of the created image so it can more easily restore each one to a clean state. At the moment, this is done on the LAVA side (in a module called lmc_utils), but it would be much better for LAVA if linaro-media-create could produce partition images directly.
Blueprint information
- Status:
- Complete
- Approver:
- None
- Priority:
- Low
- Drafter:
- Данило Шеган
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Obsolete
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
- Данило Шеган
Related branches
Sprints
Whiteboard
[2012-03-20, Danilo] Added the conversation with Zyga, will clean up and summarise later.
[2012-06-22, doanac] I might be missing something, but it seems low priority. Hopefully we'll have sd-mux to aliviate this need soon, and in the meantime we have something that's "good enough".
[2012-07-20, danilo] Killing this since it's a low priority item that we'll never get to.
<zyga> danilos, hi, could linaro-media-create output partition images instead of full card image given a special option?
<-- mertsas has quit (Ping timeout: 240 seconds)
<danilos> cjohnston, cool, I'll ping you later then (in say 4-5h)
<cjohnston> ok
<danilos> zyga, is that not easy to get from the full image anyway?
<zyga> well, harder
--> dlezcano (dlezcano@
<danilos> zyga, perhaps we can create a small utility which makes that easy (if there isn't one already), I'd rather not add to the tiny set of command line options l-m-c already has
zyga, unless this is something you find you need very frequently
--> virals (~viral@
<danilos> zyga, "you" as in you and everybody else :)
<fabo> danilos: changing mabac's approach to simply allow providing a hooks dir sufficient to solve immediate concerns. we can improve feature when more time allows it.
fabo fabo_
<zyga> danilos, right now accessing the partition means I need to loop-mount with offset that I can get while talking to parted
<danilos> fabo: excellent, thanks, we'll get it done asap in time for 2012.03 release then
<zyga> danilos, lava could use that option
<fabo> danilos: thanks a lot!
<danilos> zyga, right, but for lava, I think it'd make more sense for it to be outside l-m-c, but I am can easily be convinced otherwise; can you please file a bug, and I'll talk it over with the team to see what they think
<suihkulokki>
<-- hongbo has quit (Quit: Leaving)
<zyga> danilos, I'll start a conversation, lmc is very much essential to lava and we were hoping it could grow an option to make a master image
--> davelong (~<email address hidden>) се прикључи #linaro
<-- ryanharkin has quit (Remote host closed the connection)
<danilos> zyga, what do you mean with "master image"?
<-- omnium has quit (Remote host closed the connection)
--> mertsas (~martin@
<zyga> danilos, the master image concept that is essential for all lava work
<danilos> zyga, also, we are about to get a fix for https:/
<ubot2`> Launchpad bug 751531 in linaro-image-tools "Add support for rootfs post-processing in l-m-c" [High,In progress]
<zyga> danilos, http://
<-- Sunil_K has quit (Remote host closed the connection)
--> Sunil_K (~kvirc@
<zyga> danilos, perhaps
<danilos> zyga, thanks, definitely worthwhile to investigate having master image creation supported in l-m-c
<zyga> danilos, currently this process is manual and thus error-prone
--> pfalcon (~pfalcon@
<fabo> zyga: we talked about using Dave script to generate master image and create a ci job
<zyga> fabo: ci job?
--> ryanharkin (~ryanharki@
<fabo> zyga: ci.l.o (jenkins) job
<zyga> fabo: dave's script is far from perfect at this stage, I'm looking at how things should work
fabo: I don't see how CI is related to master images
<fabo> manual process should be avoided and master image creation automated
<zyga> fabo: anyway, master images require a "basic" image which in turns require all the knowledge l-m-c has about how to setup an SD card for a particular device
fabo: I don't see CI as a part of that process
<fabo> "automated"
<zyga> fabo: unless we want to create another entity that knows all about all hardware
fabo: the right place to keep this knowledge is lmc
fabo: I'll take automated, thank you :-)
<fabo> nothing prevent lmc in the job
<danilos> zyga, indeed, agreed
--> davidlong (~<email address hidden>) се прикључи #linaro
<-- davidlong has quit (Remote host closed the connection)
--> rogerq (~<email address hidden>) се прикључи #linaro
<fabo> zyga: though, we need to evaluate Dave script. I admit I never seen it.
<zyga> fabo: !evaluate
fabo: it's not a complete solution
fabo: I know what the script does
<danilos> zyga, fwiw, while I agree master image creation support would be good to have in l-m-c, I think per-partition images would be a detour we don't really want in l-m-c; as I said, that's just me, though :)
<zyga> fabo: it's 1/3 of the process at most
--> jluisn (~quassel@
<zyga> danilos, having per - partition images would allow lava progress beyond unreliable master images
danilos, but perhaps hooks could allow us to do that
--> davmor2 (~<email address hidden>) се прикључи #linaro
<fabo> zyga: haha https:/
it's making his way
<zyga> looking
fabo: well I'm glad we follow pep8 more
<fabo> I know but it's the 1s time we get this kind of comments... which is good imo
<danilos> zyga, what makes master images unreliable? if it's the way they are created, having it done by l-m-c should improve the situation; if not, what lava does with such images and what could it do better with per-partition images?
<zyga> danilos, a long list of things that starts with them being read-write and the fact that any rouge/acciental test can mount and change the master
danilos, if masters were always bit identical (on boot) then we'd solve a whole class of problems
danilos, thus limiting the unreliability to connection issues and bootloader environment
<zyga> (and SD cards dying but we have no way of fixing that with software)
<danilos> zyga, right, I don't see how just using full master images wouldn't solve anything that using per-partition images would: are you suggesting LAVA could more easily then restore each partition to a clean state?
<zyga> danilos, because we'd mount master rootfs via NBD
danilos, a single machine can happily serve that to a whole cluster of boards, with full snapshots and restore-
danilos, so ideally we'd have a bootloader on the card and the rest would be written over with the test image 's root filesystem
<-- sumits has quit (Quit: Leaving)
<danilos> zyga, ah, I see; basically, you'd only need the rootfs split out then, right?
<zyga> danilos, we still need to copy the kernel over somehow but I believe that the master / test kernel could also be loaded from the network
danilos, well the boot partition is also important as it has the initrd/kernel
<danilos> zyga, ok
<-- uweigand has quit (Quit: Ex-Chat)
<zyga> danilos, I think it would be best to keep the whole test image boot similar to normal use case (all from SD)
danilos, just keep the immutable boot loader in a place that minimizes damage (for example, on upgrades)
danilos, we still cannot copy the whole card image directly because of this
danilos, so lava needs to post-process each image (even if we did not call l-m-c to create it ourselves)
--> c10ud (~c10ud@
<-- gesha has quit (Remote host closed the connection)
<danilos> zyga, right, then it definitely sounds like something that'd be very useful to have in l-i-t
<zyga> danilos, we don't need it tomorrow but just having it would unlock this path
<danilos> zyga, how do you post-process it today? extract partitions directly in lava?
<zyga> danilos, yes
danilos, let me check and tell you though
<danilos> zyga, heh, you ain't getting it tomorrow either :)
zyga, no need to be specific, just wondering
<zyga> danilos, yes, we even have a module called lmc_utils ;-)
danilos, full with scripts that click "yes" for snowball
<danilos> zyga, I'll file a BP with reqs, and just paste our discussion in the notes for now, so we can expand on it later
<zyga> danilos, thanks, please subscribe me to this bp
<-- siam has quit (Ping timeout: 245 seconds)
--> cltang (~chatzilla@
gesha (~<email address hidden>) се прикључи #linaro
