Exploration of lp 1864787 - "sfdisk --force --append /dev/sda"

Development
#1

Bug report - https://bugs.launchpad.net/calamares/+bug/1864787 or calamares error " “sfdisk --force --append /dev/sda”" (recorded as impacting four total users, that could get around issue too)

@wxl recently asked for exploration of this issue.
@leok and I have talked about it, and likely best approach forward is (following is cut/pasted from elsewhere, slight editing)

Purpose of this, is to let @wxl see what is intended, it also allows others to provide insight, make comment (flaws/improvements in approaches etc) and have an easy place for documenting what is done.

As users were able to work around the issue (myself included) I don’t consider it a priority task, though with feature freeze on 25 February on hirsute…

3 Likes
#2

@guiverc great idea making this whole conversation public as it will mean others might be able to jump in and offer advise.

At some point, I need to implement our whole email system on Discourse so new topics can be emailed here. That way you can easily echo any message to the mailing list at the same time since there are people there that don’t pay attention to Discourse (and vice versa) but that’s not really relevant to this particular topic.

So I’m not sure you saw the further conversation on IRC but tl;dr Calamares’ philosophy about manual partitioning is that you proceed at your own risk. It won’t prevent you from doing something impossible (meaning here, outside the capabilities of a filesystem or partition table scheme). I think Occam’s Razor would suggest this to be the parsimonious explanation, especially in light of the complex partition scheme and the fact that manually partitioning outside of the installer didn’t seem to be any more successful.

One thing that might be helpful to help us visualize and work through this is screenshots from Calamares of what the initial scheme is like as well as what you’re trying to change it to. Perhaps that might provide some insight.

BTW I agree this is not high priority at all.

2 Likes
#3

screen
screen1680×1050 107 KB

Not of calamares; just booted a recent Lubuntu 20.04.2 (2021-01-07 daily) ISO and what I see in KDE Partition Manager. I just re-use sda8 (/) and sda9 (/home) though both are within an extended partition (sda2).

For anyone else looking, it’ll show something this box has that few these days experience… tools picking up the fd0 or 3.5" floppy device when a floppy is inserted.

3 Likes
#4

I planned to install focal twice, so may do it again tomorrow, then evaluate & confirm what I did matches the bug report; the only difference I noted was sda8 & sda9 had swapped, but that’s happened before when a partition is formatted. I’m too tired now, but assuming I made no mistakes, I’ve got nowhere in re-creating it today (or thus far).

2 Likes
#5

I decided to wait for another daily, rather than repeat with the exact same ISO the next day (where zsync showed no update)

I’ll complete the testing (so far only bionic has been confirmed perfect; & it’s a multiple OS system, but I expect all perfect, then I’ll write summary of this on bug report (last two posts here)

Anyway, this minimal testing doesn’t show it impacting hirsute or focal, and we can’t do much about groovy anyway.

I’ll use this d755-5 box now and again for the same type of installation (separate /home is a testcase of ours), so if it pops up again… I can return here.

NOTE: There are 9 references to BTRFS (4 of XFS) in the bug report (some will be repeats), maybe I need to re-try using BTRFS (and/or XFS) next time…

2 Likes
#6

Doing some QA-testing for focal.2

  • install on same d755-5 as per prior tests, using BTFS and no issue

  • install on same d755-5 as per prior tests, using XFS and problem 1864787

I don’t think XFS is the cause; I think it’s just a rare occasional issue myself… (though it could be unusual options like XFS/BTRFS/… increase chance of occurrence. If I knew a pattern long ago, I don’t know it anymore)

Install using Calamares (entire disk) in Lubuntu Desktop amd64 in Focal Daily | Ubuntu QA

reboot, restart live system & try and repeat the last install, problem 1864787 again)

reboot, used KDE partition manager to delete sda9, re-create it (XFS) then repeat install with same parameters (format sda9 as XFS, re-use sda8 as /home) & no issue.

2 Likes
#7

I’ve done a few (XFS) installs of focal.2 but on

  • sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

without issue. I didn’t expect issues, as I’ve only had this issue on the one d755(-5) box I have (and then not always).

Note: svp112a1cw is uEFI, d755 is BIOS… but I’ve not been able to re-create it on hp dc7700 either in the past; I selected the svp112a1cw as it’s faster for repeated installs

2 Likes
#8

I’m kind of getting the feeling we can’t figure out how to reproduce this. Perhaps it would be best to cut our losses and call the bug invalid. If it appears again, I vote we make for a new bug with very clear steps to reproduce and ideally the reporter would ensure that they can indeed reproduce it themselves. Thoughts?

#9

Per convo on IRC, we’ll set this as incomplete.

1 Like
#10

I could do more installs to try and find a pattern, but fear it’ll be a huge amount of work, and not achieve a lot…

I’ll instead do the occasional install on it, which if the bug occurs, will keep the incomplete bug alive, (as would other users experiencing it).

But to me, unless it occurs in hirsute where I’ve not experienced it yet, I 100% currently agree with ‘incomplete’ (reaching the thought as you typed it but before you posted at standup)

If it occurs on hirsute, I’ll start new bug report & reference (one can be marked duplicate of other)

4 Likes
#11

Looking thru our testcase checklist I noted this test was now on the older ones we have, so decided I’d run it again.

  • dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)

Again using

  • sda8 as /home (no format)
  • sda9 as / (format)

but this time I was using BRTFS (hey it worked last time, and I didn’t want issues), but failure with lp 1864787

I’d re-done XFS using another box to avoid this issue, so the BTRFS testcase was now an older one… This bug only seems to impact specific boxes, thus is really minor & likely related to a firmware imperfection/bug??

#12

If it’s specific to a particular machine, it might be good to note it somewhere but other than that, I wouldn’t sweat it much more.

#13

I’m not sweating it; I like that the checklist has loads of boxes listed in the pc.detail section, so was trying to keep that model dell in the list (by re-running the older test).

A quick test to avoid the issue is to use another box (what I did with XFS), but that pc.detail will disappear, and I think it looks good with varied hardware used, not just all tests done on a small number of boxes…

Anyway, restart focal.2 ISO (KDE partition manager just crashes otherwise after that error) & deleted the unknown partition, re-created it, then restart calamares using same options

  • sda8 as /home (no format)
  • sda9 as / (format, BTFS)

and it appears to be all good :slight_smile:

1 Like
#14

I just experienced this again on a different box

  • dell [optiplex] 755 (c2d-e8300, 8gb, amd/ati radeon rv610/radeon hd2400 pro/xt)

I’ve filed a new bug, but I’m convinced it’s a duplicate being the same issue.

I was previously convinced I had the issue only occur on the much older d755 (identical boxes from front, but different motherboard & ports)

This install was a “Install Alongside”, prior install on the box was a Xubuntu 20.04.2 QA-test I believe, but even if it wasn’t, the only other alternative is it was a Lubuntu 20.04.2 as that’s all I’ve done in recent days.

#15

Bad news, I just experienced 1864787 many times in hirsute. On the original d755-5 box, I had three fails (whenever I had calamares format the partition; XFS but I don’t think XFS is the issue)

  • dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)

The partition was a valid partition before hand, BTRFS (prior install & empty), REISERFS (empty) before starting.

I’ll just provide link to the QA-test page, but I’ve uploaded some ~/.cache/calamares/session.log files to the 1864787 bug report…

I got tired at that point, so next install had me use KDE Partition Manager to format exactly what I wanted (XFS) and just select the partition (ie. identical install conditions except it was KEEP instead of FORMAT) and it worked as is usual.

Disappointing in that this is the first time I’ve had this issue with hirsute… however it’s still easy to work around it.

1 Like
#16

Repeat test of yesterday’s and 1864787 again.

Same box, but using BRTFS today

  • dell [optiplex] 755 (c2d-e6850, 5gb, amd/ati radeon rv516/x1300/x1550)

Only tried a single install (with format) which failed, then rebooted & as with last time, used KDE Partition Manager to format unknown partition to BTRFS, then repeated install except this time I used “KEEP” (instead of ‘format’) and it worked.

I didn’t upload anything to launchpad, but as today I’m also testing mkusb persistence I may have some details on the installation media if helpful (I don’t believe there is anything useful there, and I believe the mkusb media does not impact/influence this; yesterday was not a persistent image)

I may continue repeating this install daily, or every so often & log here… for pattern purposes at least.

http://iso.qa.ubuntu.com/qatracker/milestones/419/builds/226913/testcases/1701/results

I couldn’t recall how I wrote yesterday’s ISO to media (I just know it wasn’t persistence), so I re-ran today’s test, switching to XFS, but with media written with the command

sudo dd bs=4M oflag=sync status=progress of=/dev/sdb if=/de2900/lubuntu_64/hirsute-desktop-amd64.iso

and again failure when formatting… I’ll now reboot & use KDE partition manager to prepare partition & just ‘keep’ (instead of ‘format’) and I expect it to succeed.

#17

My last install (yesterday) in exploration of this issue failed… reported under error Bug #1917724 “lubuntu hirsute install fail mkfs.xfs” : Bugs : calamares package : Ubuntu

My normal fixes also failed with KDE Partition Manager dying too (https://bugs.launchpad.net/bugs/1917753)

… but I need to get that box box fixed so I’ll try again with today’s hirsute daily and if I have issues, I’ll get it resolved using gparted (or focal if needs be) and do some testing on other boxes… as per Comment #4 : Bug #1917724 : Bugs : calamares package : Ubuntu

–
Update: I booted today’s hirsute daily and had no issues with removing the UNKNOWN partition & formatting it reiserfs; no repeat today of the https://bugs.launchpad.net/bugs/1917753 that I experienced twice yesterday (fail; reboot & try same stuff again & fail again yesterday…; though I did the same today & all good :slight_smile: )

I prepared the partition (sda9 for /) in KDE Partition Manager, and then just selected /home sda8 & and / sda9 without format and install worked fine. The normal work-around was flawless today.

1 Like
#18

Another install on d755-5

  • reuse sda8 as /home, no-format (ext4)
  • format sda9 (was reiserfs) XFS

I’ve tagged it as Bug #1917724 “lubuntu hirsute install fail mkfs.xfs” : Bugs : calamares package : Ubuntu as that message is what I get.

That bug had been marked ‘incomplete’ so I’ve changed it to ‘new’. However (as I left in comment on bug report)

I am CONVINCED this issue relates to 1864787 & 1914521 and is likely(99.9%) a DUPLICATE (just a different message)

I don’t consider this bug is worth of dev time given that is a limited resource.

This issue won’t impact many users; I’ve been trying to consider how many, and I’d say I see maybe 5-9 of these per cycle (ie. focal or groovy). It’s easy to direct them to manually setup their partitions & tell them to have calamares to just use (no-format) pre-prepared partitions (how I’ll fix this box; using reiserfs so the partition stands out next test). Maybe 5-9 may qualify as a note in the release notes like we had in focal being worthwhile.

Note: the 5-9 is a guess; it’s likely more for focal and fewer for groovy…

1 Like
#19

I just did a final QA-test looking for this issue…, same d755-5 with the usual

  • reuse sda8 as /home (no-format, ext4)
  • format sda9 as / (was reiserfs) XFS today

All clean… (though I’m still not 100% convinced I didn’t do something wrong… a bug was expected on this install)

I’m not sure a single good install is enough to close the bug report yet (I’ve had the odd good install before on specific daily images and then have it return on next daily), but it is good enough for me to recommend no mention of the issue in release notes.

The issue occurs only on rare boxes, with specific partition schemes so it impacts very few anyway. Had the issue occurred I’d still not have cared if it was mentioned or not (easily dealt with via support; ie. manually partition as wanted before running calamares). If a new RC is spun, I’ll likely re-test (BTRFS next time)

1 Like
#20

Another QA-test using same d755-5

  • reuse sda8 as /home (no-format, ext4)
  • format sda9 as / (reiserfs, @leok had already done a BTRFS today which I usually alternate with XFS hence used reiserfs)

All clean again

(looks anyway; as I’m still running tests before I close the report on iso.qa.ubu but I’ll return if issues are found)

2 Likes