In T367536#10076979, @Epantaleo wrote:

Hello, I would like the VM etytree-a to be kept as a copy, as I am planning an upgrade of the operating system, but I need the old copy as a reference to build the updated one. The issue is that I use Openlink Virtuoso as a database and I need time to figure out how to manage the upgrade. @fnegri kindly helped me but I did not find the time to proceed. I think by the end of October I will have sorted the issue out and I will create and update version of the etytree application. Thanks!

Ester writes:

These are now rebuilt with proper partitioning. They probably shouldn't be bootstrapped until T372821 is resolved.

All three of these need reimaging to get the drive labels set up properly; right now they all have a big OSD drive assigned to the os.

@cmooney says about cloudcephosd1036:

As the last custodians of the wikilabels infra, we/I can confirm that it
is unused and can be deleted. All the code that might be relevant is
archived on GH or elsewhere, so if there was any _historical_ interest,
that can be dug up if ever needed.

Best,
Tobias

I created a new Bookworm VM and moved the cinder volume over. I expected to be able to launch striker with 'docker compose' but...

I don't see evidence that there has been any progress with this project. Please respond with a plan and timeline if you would like me to not delete these VMs.

emailed today:

Emailed today:

Emailed today, again:

Thanks @EgonWillighagen. This project is now up to date.

I've now repooled all affected ceph nodes (and rebuilt cloudcephosd1035) and repooled all cloudvirts. Until the switch flakes out again this is resolved! thx @cmooney @dcaro @VRiley-WMF

(meanwhile I am draining and rebuilding cloudcephosd1035 because it was built with improper drive assignments.)

@cmooney can we get cloudcephosd1036 set up now that the switch work is done?

Quick summary:

Hi folks! I haven't read all of this ticket but can I get an update about when/if you plan to remove the Buster VMs in this project?

I'm going to delete these VMs next week. If you need to check them or rescue an data, now's the time :)

Emailed project admins today:

Hi @Beetstra! Did you ever get your access issues sorted out? I need to rebuild the host that these VMs are on soon (see T364457 for probably-uninteresting context).

Hello @Tgr, were you able to make any progress with this? I'm going on sabbatical soon and (for arcane backend reasons, T364457) need to clean up the host these are on before then.

+1 workaround ridiculous bug

In T370460#10057924, @Jgiannelos wrote:

Any idea who can help with that ?

On 2024-08-05 I sent this email to the releng and sre mailing lists:

codfw1dev is now running C

Current plan:

The remaining case is in wmf_sink, the proxy cleanup code. That code relies on being able to look up the dns records (and, by extension, the IPs) associated with a deleted VM. That is no longer possible.

It's a new week and a new nag! @Tgr if you would like to prevent these VMs from being shut down please upgrade them soon. Thanks!

@notconfusing can you confirm that you still plan to work on this?

OK, done.

Oh, good news, I already did this all but one (possibly tricky) case.

Oh, great. @Southparkfan does that unblock you here?

deleted!

Oh, I see that the prod poolcounter hosts are also still running Buster. Is someone tasked with fixing that?

Now we have per-service users for each openstack service.

@Dzahn iirc you were able to do some reprepro magic recently? If not I can give it a go but it's been a LONG time

I added a bunch of openstack things, and removed the grid engine reference. Now I'm passing this over to @dcaro to add toolforge dependencies, e.g. harbor

No response so I've emptied this directory.

@cmooney can you advise what (if anything) needs doing here?

Your VM now has a /data/scratch mountpoint. New future VMs should have it as well, thanks to a project-wide hiera setting I added in horizon.

Made the same change for the eqiad1 logs

Puppet is failing on the new host because of

thank you!

@Jgiannelos do you have any suggestions about this ticket? Shall I just delete restbase04 and let the chips fall where they may?

Thank you for paying attention to this, @Tgr. Do you still hope to work on this transfer?

Closing this for now because it doesn't seem to be happening repeatedly.

In T371356#10029387, @Andrew wrote:

Would this be best addressed by changing the default logstash board settings to filter info and debug logs? Then they'd be there if we needed them.

Would this be best addressed by changing the default logstash board settings to filter info and debug logs? Then they'd be there if we needed them.

Thanks @dcaro!

I did a full reset and rebuild of rabbitmq. I definitely do not know why that helped :(

I removed those two old servers via the database. I don't know why the CLI doesn't work (the cloudvirts were empty).

Everything is now based off of trove-guest-bobcat-ubuntu-jammy. That image contains a local application of proposed patch https://review.opendev.org/c/openstack/trove/+/924285

Can you tell me what (if any) change or update preceded this?

The best procedure is to not re-use hostnames (typically when there are names like host05 we would just increment and name the next one host06). If you need to re-use names, wait a few minutes after deleting before recreating.

done. thanks for doing this work!

Hi! I'm very sorry this got missed; would you still like /scratch added to your cloud-vps project? Or did that get done already?

Long since done and migrated

Seems moot at this point, we're running cloud-vps on Bookworm now

I'll see if I can wrap this up