Just in case anyone is using their account here to post to off-instance communities: those posts and comments seem to have a very high failure rate. There is a lot of activity and accounts on this instance. This is to raise awareness, not to pull people away or break up the lemmyverse. Quite the opposite really: there is a technical problem on this instance that might be preventing the lemmyverse from functioning as it should.
deleted by creator
Lemmy.world and some others have held back on the 0.18.0 back end update because the admin did not want to forego this Captcha integration, which broke in 0.18.0. Version 0.18.1 is expected to release soon, and address the regression with Captcha.
The new version also reduces its reliance on websockete, which should address a number of other quality-of-life problems on this instance, such as the random post bug (and hopefully the 404s/JSON errors I’ve been getting all afternoon).
Hopefully just a few more days and we’ll be back to rights.
The new version also reduces its reliance on websockete
From my understaind it’s not a reduction, but a complete replacement.
You’re probably right. I’m not a programmer and am not familiar with the code base so I was avoiding speaking in absolutes.
Sith vibe check passed
Come to.the dark side. We have cookies!
An aside, the Astros are up 2-1 in the bottom of the 3rd.
🤣
I don’t think it’s a version issue. It could be, but testing I’ve done says otherwise. There’s plenty of 0.17.4 instances that, despite other bugs, have no problem sending and receiving updates from other instances. I hope you’re right. I don’t run the instance so testing reveals mostly speculation.
There’s plenty of 0.17.4 instances that, despite other bugs, have no problem sending and receiving updates from other instances.
It’s probably because of the absolute scale of dot world. This is what happens when you try to centralize ActivityPub, especially an implementation like Lemmy which did not previously have any of the scaling challenges the likes of Mastodon had before.
Everyone should’ve told people to pick different instances (there are STILL new guides written where step 1 is to “just register on dot world” [or shitjustworks, which isn’t any better, really]) and admin should’ve locked registrations after 10-15k, maybe 20k users MAX, yet y’all got too greedy. Good fucking luck dealing with the aftermath.
I should probably go to bed, I’m getting grumpy.
In my opinion, we need to somehow solve the community centralization issue first. MultiCommunities, or some way to aggregate the dozens of large-ish groups like “news, technology, etc” and be able to subscribe to all of them in one fell swoop would allow people to spread out to other instances much more reliably.
I’ve brought this up as a suggestion elsewhere. People seem annoyed at the lemmy vs kbin idea of “communities vs magazines”. Maybe everything is changed to “communities” and “magazines” are officially adopted as community-maintained mega-lists of common communities.
An example. There’s a bunch of car manufacturers. Sure, maybe I could just select the “Honda” community on every instance I can find, or instead I subscribe to the magazine called “Honda” which auto-subscribes me to every single Honda community in the list… or even the magazine called “Cars” which would include all manufacturers and cars communities.
Then there could be a Magazine view for that Magazine which would allow all posts from those communities to be aggregated in one place.
Just spitballing ideas.
That is an issue for sure, but probably better addressed at the ActivityPub level? IMO, an instance should be limited to a single community.
Yes, this would be a change to the underlying protocol. I think it’s definitely worth discussing how this problem can be solved by people who maintain ActivityPub. IMHO the Lemmy and kbin developers should also be a part of these discussions as well.
I don’t think it can be fixed without scaling up the instances, which isn’t for the faint of sysadmin heart, even if you CAN afford the hosting.
What do you think is causing the failures? How are you seeing the fact that it’s failing?
Like you mentioned afterward. Comments and posts just plain failing to land on any other instance. Also I run an instance for testing and can see incoming connections. lemmy.world fails at a protocol level, not at the application level. It’s a, IMHO, bandwidth issue. Hopefully the admin is aware and wants to fix it. I’d say he has a responsibility, but he doesn’t. lol.
So I’ve been thinking of spinning up a personal instance. I have the tech chops for most projects like this in a small scale but are there any reasons why I shouldn’t use Lemmy in that way?
Do it. Be super aware of how federation works though. The more stuff your users subscribe to the more network traffic you will need to accommodate.
Is there any use to spinning up an instance, only allowing say 10 people max, then just keep it updated and let it run, to take the load of those ten people off the bigger instances? Is that too small time to be useful? I have pretty weak upload.
I think that would be worth it, yeah. The way I see it is that some instances are going to host the largest communities, and therefore those instances are going to need to handle all of the incoming/outgoing updates to posts in those communities. Right now they can’t do that reliably and push updates out to all of their users’ devices.
So in the long run I think having small/medium instances (say a couple hundred, not tens of thousands of users) will be the way to grow. These smaller communities can push updates to their smaller user count reliably, and then have more resources to handle federated content coming in and going out.
Upload will depend on your user’s activity (posting/voting/commenting.) Download will depend on that PLUS what the users are subscribed to.
Just trying to get a sense if it’s worth bothering with my 10Mbit up.
It’s really quick to get running. You need a fairly reliable IP/hostname as this is how your server will talk to all other servers and users. If you use dynamic dns, make sure it’s rapidly updated if your ip changes.
Yeah, I just don’t want to set it up, realize it’s trashing my home connection, and then have to boot off those ten users. Maybe I’ll set one up with just me, then if everything is alright, add one more and see. Thanks for the tips! :)
That’s what I am doing to suss it out. I’m a data nerd so I’m looking carefully at how to contribute and watching what’s going on sub-meta…
This is the problem we are having with fast growth on a few select communities. The largest servers are being bogged down simply because the software has not been tuned for these large types of instances yet. ActivityPub works best (in it’s current state) by spreading users over smaller/medium sized instances. Folks need to take a look at other instances (and I agree it is hard to find them for a newcomer). You can look at https://fedidb.org/ to look at instances that have been indexed running kbin, lemmy, and other software.
Joining a smaller instance means that your server is not being bogged down by tens of thousands of other users trying to pull updates to their devices at the same time. You can still see the content from other instances, and in many cases it is more reliable because your smaller instance actually has the resources to handle pulling in the posts you want to see. The server-to-server communications that make content federation possible are less resource intensive than pushing updates to user devices. Less users on an instance = more server resources to actually federate content. In the future I am sure instances like lemmy.world will be able to handle the user traffic and federation traffic smoothly, but for now the best way to ensure stability is to join a smaller instance.
(Plug for my instance: https://remy.city, a general purpose Kbin instance. I set it up for personal use but anyone is free to join me in using it. I have defederated from the instances with more extreme viewpoints userbase-wide (like lemmygrad and exploding-heads), and from lemmynsfw.com because of content hosting concerns. I’m open to suggestions on others.)
First, thanks for putting your instance out there! We need more open servers.
Second, lemmygrad is straight-up Marxist, which is about as far left as you can get - pretty much the opposite of “alt-right”.
You are totally right and my brain definitely farted on that one. Extreme is extreme to me I guess. I’ll edit to reflect that.
It’s a known issue:
https://github.com/LemmyNet/lemmy/issues/3101
Seems to be growing pains from Lemmy having to scale up in a very short amount of time so hopefully they are able to find a workaround soon.
It’s not happening everywhere and all the time but it happens enough for you to think a post or comment has no engagement when it does just not on your instance.
It’s a known issue, but if you look at the issues, the reason for it is “could be lots of things.” Relying on a FOSS project to fix the problems of the community is kind of naive. If the admins don’t step up then I’ve no sympathy for them when the whole thing crumbles.
We really need some more developers… I wish I could really help, but it would take way more time than I have to do more than poke around.
Why did this get downvoted? Something being an “issue” on GitHub is about as meaningless as something can be. Unengaged developers are usually the result of unengaged users. I’m saying IF your admin (not that they did,) submitted an issue and then is just sitting back and waiting, that’s shitty.
That isn’t at all what is happening, the world admin team has been very engaged with the project on GitHub with a ton of back and forth and various code pushes to fix the captcha pullback that’s in 0.18. The issue you are seeing is known and the belief is it should get better with the update to 0.18.1 that the devs have said is coming in the next few days. It seems to be partly due to the size of the world instance and the problems with web sockets. You’re likely being downvoted for appearing to be authoritative about this and blaming the world admin team (it’s more than a single person) when they know this is a symptom and have a plan they think will fix it that a little more digging into the history of why they’ve chosen to forgo the 0.18.0 update would have answered your questions.
I was meaning to be authoritative, and I am low-key blaming the admin team. I grant them the benefit of the doubt though. I still don’t think it has anything to do with the 0.18.0 upgrade.
EDIT: I don’t care what the admin team does, how loyal or dedicated they are, or anything like that. I just want the reddit alternative to not suck.
I find everything to show up pretty well. Have an example?
Yeah, take a look at my profile from the perspective of lemmy.world:
Then compare with the comment count compared to other instances:
https://lemmy.ca/u/liara@lemmy.world
https://lemm.ee/u/liara@lemmy.world
9 of my comments haven’t federated and are visible only to lemmy.world
https://lemmy.ml/u/liara@lemmy.world
22 comments made it to .ml but that’s still missing 6 comments
A post I sent 4 hours ago to lemmy.ml wasn’t showing up. After I edited it once with no changes, it showed up as posted 1 minute ago. It’s definitely an issue taking place. You can check to see if your posts are showing up at their destinations with the rainbow federation link button next to posts.
This post can be an example. Check: https://sh.itjust.works/post/446063 Do you see our conversation?
What is this supposed to prove? I see the same thing there and on lemmy.world
Just to be clear: if you load these two pages: https://sh.itjust.works/post/446063 and https://lemmy.world/post/609080 that are identical?
My bad, I’m an idiot lol
No, they’re not. I was only comparing that conversation because I thought it was a direct link to that comment chain, but in fact many other comments are missing.
No you’re not. It’s good to test. There might be differences in our client’s view due to other CDNs, proxies, caching etc. I’m just convinced it’s lemmy.world’s outbound networking. There might be other instances with other problems, and yes the upgrade could be a factor, but the lemmy.world issue is clearly and forwardly apparent.
I had no idea about this issue with LW; thanks for bringing it up.
Do you know if they eventually show up at some point? Or is the request blocked the first time around and there are no other attempts?
By the way, I’ve checked on kbin.social and it looks even more different there lol
A lemmy developer more familiar would have to answer that. It seems inconsistent at best. How consistent it was designed to be might be up for debate, but the average users understanding should AT LEAST match that…
I can see a LOT of failed network connections from lemmy.world to my instance and results of comparing posts. That’s the data I have. :)
I saw this on lemmy.world, went to reply to a comment, realized I wasn’t on my instance, flipped back and I can’t find it, so sadly I fear you might be correct that there is something wrong.
Also keep in mind that defederating (blocked instances) will prevent posts and comments from syncing between instances as well.
You can see blocked instances on lemmy.world here: https://lemmy.world/instances
Correct. I am not testing with any of those instances. You shouldn’t expect that to work at all, but I guess some users might not know.
This isn’t a question
This has been discussed in an issues thread which @ruud@lemmy.world responded to, so we can assume he’s aware, though I haven’t seen an update.
Fair enough. Keep in mind, being popular doesn’t magically bestow the knowledge, resources, or willingness to handle issues.
Good point. Nothing against the larger instance owners of course. If my little instance got super popular somehow (like being recommended in guides on how to join lemmy/kbin), and thousands of users got in per day, I could see issues happening just like this. I don’t know the ins and outs of tuning this software for performance at scale, and I know I couldn’t learn it fast enough if my instance faced very fast growth like lemmy.world has.
I think admins are going to need to turn registrations off periodically, as they scale their hardware (and their knowledge) to run it for more users effectively.
being popular doesn’t magically bestow the knowledge, resources, or willingness to handle issues
I would like that on a T-Shirt, please.
I’m on kbin.social and I can see this ;)
Hilarious. This comment doesn’t show on sh.itjust.works, which makes total sense. lemmy.world is responsible for sending it to other federated servers. Maybe kbin > lemmy right now?
Soooo sh.itdoesntwork?
sh.it.works.sometimes
Lemmy.ca chcecking in. I had to scroll quite a ways to find out what instance you were complaining about.
OP, This highlights something else - I think the reason you are getting down votes is that most people seeing this thread are not on lemmy.world. The heading is a bit misleading (or potentially wrong) from the perspective of a federated user - which is most of us!
OMG. You’re right! I just edited the title. Which, lol, will not show up everywhere.
I honestly hope it’s a short term problem, I had no idea. It’d be a pain to have to migrate somewhere else.
It’ll get fixed. Right now this may be a swamped and baffled community, but it is also excited and highly motivated.
(ignore me) test post from sh.itjust.works account
deleted by creator
Yeah this is so bizarre. Is it possible that sh.itjust.works is having trouble receiving, or is it confirmed that lemmy.world is dropping the ball on outgoing notifications?
Look. I’m not pointing fingers, but I know a thing or two and I can say with a high degree of certainty it is lemmy.world. :) Does the admin care? Do the users? Who knows. I just thought it was important information to have.
The silent failures are what actually miff me the most. Like to an uneducated user, everything is just working fine, but in reality they are stuck in lemmy.world.
