Yeah we added another admin onto the team

cant add for a bit unless the user has a post or comment

should have access to my script to do it without that soon (just in exams atm)

Yeah it uses a system similar to github actions

The forgejo instance here is meant for repositories related to programming.dev and the fediverse

Cleared all the spam repos so you shouldnt see any more, will do a more permanent solution later (will see if I can do an application process using a third party form system since forgejo doesnt have that built in)

Yeah I havent gone through and cleared it in a bit since there isnt much of a point to due to it being mostly used internally only rn (and forgejo makes the process very annoying unless you do it directly through the db)

Ill purge things and adjust settings again soon

I couldnt find anything in the post details from looking at it that would differentiate it from other posts

Cant find anything that would cause this in the db. Thought the dates got messed up in federation but they seem to be perfectly normal. Still pinned even if I manually change the date to a year in the past

Im assuming something got messed up from the nodebb->lemmy federation

Im going to remove the post as thats the only way I see to get rid of it

I think the advantages of multiple communities outweighs the advantages of consolidating. Especially since things can be cross posted between the multiple communities easily

No longer able to access the content is referring to the federation. And the difficult to move off is referring to for example communities that have been attempting to get traction to move off of lemmy.ml

Not posting in the community doesn’t mean people there don’t interact with it. I have beehaw as an example on hand but im sure there’s other federation examples

I think its the kind of subject matter that fits programming.dev well and relying on outside instances for programming content with no mirror on our own site makes us too reliant on those other instances if anything happens in the future (e.g. extreme case but if that instance goes down. Lemmy handles it terribly since the community still exists as a ghost community with no federation but still viewable)

With similar logic we have a lot of the same communities as lemmy.ml communities including programmer_humor, opensource, etc. To give people an alternative spot to the lemmy.ml communities and so that we aren’t overly reliant on other infrastructure we can’t control within our instances subject matter

We typically dont close communities on programming.dev since then theres only one option for things

Ends up having things like people who use the community no longer being able to access the content and being difficult for the community to move off of it if something happens

e.g. beehaw.org is defederated from sh.itjust.works so any beehaw users wouldnt be able to use the community if the programming.dev one is closed

!emacs@programming.dev exists already

Ill go through and remove the hides from the communities that dont have bot content anymore

Not possible in lemmy atm past just rehiding everything from the instance every so often

edit: just did another round of hiding things from lemmynsfw, should be reflected in the hidden communities doc

Theres technically a code review community at !code_review@programming.dev although ive seen some people also post code reviews in the language/engine communities

we also host an instance of opengist at https://blocks.programming.dev/ as an alternative to those if wanted

Ah yeah I need to refresh the data, ill do that later

Nobody reported it as down to me, I can bring it back up

Been working on some other projects recently so havent really looked at that site much

We just added a couple more people to have server access and ideally that situation wont happen again

That problem specifially could only be reproduced on the main server so fixes couldnt be tested without taking the entire site down and at some point after the 0.19.5 upgrade (when the issue happened) it became too late to roll the server back since there was too many activities that came in but if it happens again after an upgrade we would roll back pretty much instantly)

dbzer0 does seem like a better option for general privacy discussion though

Sites back. I reindexed every table although the only one that seemed to be affected by the issues was the comment table (it had duplicates I had to remove similar to what I did for the post and community tables before)

Site will slowly catch up in activities over the next couple hours so youll see posts, etc. that were made in the few hours it was down slowly come in

No votes or comments were connected to the duplicates but yeah if there was some you would move it over to the other one

The posts that had comments were the posts that were uniquely on the other community and those moved over with the post when I updated the community of the post

Sites back, communities that previously had their community page not vieweable are now viewable

An example of this is https://programming.dev/c/a11y

Basically what I did was before the maintenance exported every single post in the post table and use a js script to check for duplicate ap_ids (checking using sql itself didnt work due to broken indices so it was always missing some). Then I generated sql statements to remove duplicate posts that had higher ids than the other posts theyre a duplicate of. Ran all those during maintenance and since all duplicate ap_ids were removed I could now reindex the table to fix the broken indices

Did the same for communities after the posts (but also did some statements to move posts from one version of the community to the other)

Site might be slow for a tiny bit as it processes the 2 hour of activities that were generated while it was down

Sites back up

Got a lot of information about what it actually causing the issues which should help me automate some fixes for tomorrow + the weekend

There likely will be some more downtime tomorrow if I have time (if not itll be on saturday)

The issue is essentially a chain of issues with one needing to be fixed before I can fix the others. Some tables have duplicates entries which is preventing reindexing (since the indexes are broken). First one im going to try to fix is the post table.

I did some changes to remove a duplicate community to test which is https://programming.dev/c/photography@lemmy.world and ive managed to make it merged and now comments will show if you sort by comments in the community. Posts however still aren’t showing up due to the posts table having the issue with duplicates as well. (fixing posts first so I can set all of the posts to the correct community in an automated way later for the communities)