For a high-level introduction to data federation, as well as a comparison to other federated social protocols, check out the Bluesky blog.

cross-posted from: https://discuss.online/post/5484255

February 22, 2024 Bluesky writes:

Up until now, every user on the network used a Bluesky PDS (Personal Data Server) to host their data. We’ve already federated our own data hosting on the backend, both to help operationally scale our service, and to prove out the technical underpinnings of an openly federated network. But today we’re opening up federation for anyone else to begin connecting with the network.

The PDS, in many ways, fulfills a simple role: it hosts your account and gives you the ability to log in, it holds the signing keys for your data, and it keeps your data online and highly available. Unlike a Mastodon instance, it does not need to function as a full-fledged social media service. We wanted to make atproto data hosting—like web hosting—into a fairly simple commoditized service. The PDS’s role has been limited in scope to achieve this goal. By limiting the scope, the role of a PDS in maintaining an open and fluid data network has become all the more powerful.

We’ve packaged the PDS into a friendly distribution with an installer script that handles much of the complexity of setting up a PDS. After you set up your PDS and join the PDS Admins Discord to submit a request for your PDS to be added to the network, your PDS’s data will get routed to other services in the network (like feed generators and the Bluesky Appview) through our Relay, the firehose provider. Check out our Federation Overview for more information on how data flows through the atproto network.

Read Early Access Federation for Self-Hosters

    • ericjmorey@discuss.onlineOPEnglish
      5·
      9 months ago

      It will be interesting to see what Friendica devs come up with!

      I’ve just started looking at the AT protocol. What sort of WTF things are in there?

      • owls@community.yshi.org
        1·
        9 months ago

        not the OP, but

        i was going to point to the did:plc thing they made up and went with. but since the last time i looked, it looks like they support (and prefer) did:web, so that’s sorted out.

        the “wtf” i have is more to do with actually running a community with atproto. you need a central crawler service that knows about all the PDSes you want to be friends with (this is presumably why you need to sign up in their discord right now, they gotta tell their crawler to look at your PDS)

        but i think the crawler has to grab the entire PDS for everyone it knows about? if you want a large community, it seems pretty resource-intensive since the ceiling is “infinity posts”. bsky’s open source code suggests they have chosen not to deal with this problem.

        with most AP services (e.g. mastodon), you can prune the data and the only consequence is “you don’t get full text search for super old posts received from other services that we pruned”, so there are ways to limit the cost other than “limit the number of users in our community”.

        but this may just be an implementation detail and not an issue with atproto, e.g. git shallow clones are a thing, and the PDS is also storing a big merkel tree. i am not sure if the indexer relies on having the complete history or not (since you do need it for certain operations). bsky’s own code just shrugging suggests maybe limiting it is challenging, i dunno.

        • Elle@lemmy.worldEnglish
          2·
          9 months ago

          What you note about the crawler appears to be essentially by design:

          The federation architecture allows anyone to host a Relay, though it’s a fairly resource-demanding service. In all likelihood, there may be a few large full-network providers, and then a long tail of partial-network providers. Small bespoke Relays could also service tightly or well-defined slices of the network, like a specific new application or a small community.

          In their section on so-called “Big World” design:

          As a result, we opted to architect the protocol in a “big world with small world fallbacks” way. With the web, individual computers upload content to the network, and then all of that content is then broadcasted back to other computers. Similarly, with the AT Protocol, we’re sending messages to a much smaller number of big aggregators, which then broadcast that data to personal data servers across the network.

          Emphasis mine.

    • Flaky@iusearchlinux.fyiEnglish
      5·
      9 months ago

      Yeah, from some cursory glances and following of AT devs, some things I understand the logic of and some things I’m thinking “isn’t this a bit over-engineered?”

      • dumpsterlid@lemmy.worldEnglish
        1·
        9 months ago

        My basic question is whether the problems that needed solving about Twitter and other corporate social networks really have anything to do with software engineering at the end of the day.

        Writing the code is honestly the easiest part in all probability. It is the other parts like community ownership, resiliency against corporate capture and human moderation that takes into consideration complex contexts etc… that are the truly hard parts and Bluesky really says almost nothing (at least worth trusting as more than words) new along those vital metrics (chiefly because it is an investor backed for-profit venture).

        • Flaky@iusearchlinux.fyiEnglish
          0·
          9 months ago

          I think the moderation will be an uphill battle for Bluesky. I haven’t seen a clear answer over how legal issues are going to be handled and generally, people want some form of moderation. Maybe not the extent that the fediverse has with its blocking drama.

          But the resiliency against corporate capture and community ownership, meh I’m not really worried. I work with and use open-source projects that have been backed by corporations, Mastodon.social has already said they wish to federate with Threads, and there are corporations sponsoring (in the case of mastodonapp.uk) or outright owning instances (in the case of Flipboard, Mozilla Social and Vivaldi Social). Bluesky seems to be built on the notion that it too will be a possible adversary in the future, so the protocol is being built with that in mind.

          • dumpsterlid@lemmy.worldEnglish
            1·
            9 months ago

            But the resiliency against corporate capture and community ownership, meh I’m not really worried.

            You have to explain to me why you think massive tech corporations are going to behave differently here than they always do. Every large tech company behaves like Microsoft after a certain size in terms of values and actions, and they will do their best to mine the valuable aspects of the fediverse out and silo them away in a way that can be monetized.

            I consider Flipboard or Mozilla owning instances to be a far different question because these are relatively small corporations, they aren’t Meta, they don’t have more cash on hand than entire countries.

            I think the moderation will be an uphill battle for Bluesky.

            Moderation is the hard part about social media, who gets to moderate, how moderation is handled between communities and how much human moderation genuinely happens from within the context of communities are all the important questions.

            Again, what happens when Bluesky’s investor’s come knocking and want monetization? At that point is the CEO really just going to say “we can’t do that, it would give us more profits but it would be wrong to undercut the openness of Bluesky!”. It is frankly ridiculous to assume this would happen, the same story will play out that always plays out here.

            You can either make huge amounts of money off of social media and payback your investors or you can make a healthy community, pick one. Unless you are a massive corporation with a lottttt of investors to please, then there was never really a choice no matter how long your investors let you attempt to fool your customers into thinking so before you hit the gas on cashing in (Reddit).

            • Flaky@iusearchlinux.fyiEnglish
              2·
              9 months ago

              This all implies Bluesky can be considered a massive tech corp (which it honestly isn’t even with investors, definitely not compared to Meta, or even Mozilla at this point), and can even be monetised.

              In the event that they do attempt that, users can move to a different PDS and not lose any of their data - that’s how AT was built. While on AP, it’s dependent on if the software powering the account supports migration, and even then I’ve not seen an implementation that carries over all of the user’s data (Mastodon only does followers/following, Lemmy has no migration whatsoever). That’s not to say it’s impossible, but I’ve not seen it happen.

              • dumpsterlid@lemmy.worldEnglish
                0·
                9 months ago

                But what is the business model here? Can you honestly tell me with a straight face that you trust them?

                After a certain point I don’t care about technical arguments about how the AT protocol is better than AP or how technical aspects of it make it impervious to being controlled, there are always ways. The way we stop it is political, not technical, and just trusting Bluesky will be benevolent towards its federated users indefinitely is not a winning political strategy here in terms of the power dynamics between the ruling class (people like Jack Dorsey) and the rest of us.

                Further I bet most of the people involved in Bluesky are really passionate and genuine in their desire, but ultimately their good intentions don’t interface at all with the reality of the structure they are operating in, i.e. an investor backed corporation seeking to profit off of a social network backed by some of the richest most powerful people in the tech world (even if they haven’t directly poured huge amounts of capital into it, they could, the option is there).

                You have to ask, why the hell would they be genuinely offering us the future we want for the fediverse? It makes no sense for them too since they simply stand to lose by creating that future for us.

                • Flaky@iusearchlinux.fyiEnglish
                  0·
                  9 months ago

                  Bluesky’s development has been pretty democratic, moreso than Mastodon where one guy basically leads the entire trajectory of the fediverse (at least from a mainstream perspective) from what I heard.

                  As for Jack, hasn’t been on Bluesky for a while now - he prefers Nostr. Even though he’s on the board, he’s not attended any meetings nor has he dictated how the platform should go. He just threw money at it and ran after the community didn’t take him seriously.