I have a lemmy account on lemmy.sdf.org

When searching for communities to join I can see several in the list from lemmy.ml and beehaw.org

but while searching for some communities that I know exist on lemmy.ml or beehaw.org I am getting no results

will a community from instance A only appear in search results on instance B if a user from instance B has already manually subscribed to that specific community from instance A?

basically, there is no scraping of ‘all available’ communities even when two instances know of each others existence? only ‘specific communities’ are indexed when a user has manually found/subscribed?

    • lackthought@lemmy.sdf.orgOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      that lemmy.directory looks great!

      a minor issue with this workaround is that when I search for a community they all show 1 subscriber (your scraper obviously) and no more information about the true size of the community, you have to click on them to see which ones may or may not have user activity

      so if the original design of lemmy is to only index previously subscribed communities it will be difficult for smaller/niche communities to grow and become discovered (unless someone uses your directory obviously)

      hopefully the standard lemmy search can be changed so it is more universal by default and with better metrics rather than just # of subscribers from the current instance

      • kiwi@kale.social
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        1 year ago

        Thanks for the feedback! I think if your main goal is to just search for communities by name, then it’s best to use: https://browse.feddit.de/.

        I find lemmy.directory the most helpful in browsing all posts across the lemmyverse in a single feed (via the “All” filter).

        Searching for new communities may be able to be made more universal, but automatically pulling in those feeds (subscribing) is probably not a good general idea. See here for more info on that: https://lemmy.ml/comment/476925

        • lackthought@lemmy.sdf.orgOP
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          how does your directory (or the search from feddit.de for that matter) find brand new communities?

          suppose I create a community on sdf.org called ‘HooverMaxExtractPressureProModel60’ for discussion on a specific vacuum cleaner

          how will any 3rd party scraper/search tool know of its existence so it can be indexed?

          I was thinking each instance could keep of a list of all their own communities and share that list with other known instances in some way as a way to ‘spread the word’ on what is available

          • kiwi@kale.social
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            I think that would be possible to do to help with searching for new communities. Although for now I’m not sure there’s too much difference between it existing on each instance vs on https://browse.feddit.de/. Other than ease of use of staying on one website I guess.

            • lackthought@lemmy.sdf.orgOP
              link
              fedilink
              English
              arrow-up
              2
              ·
              1 year ago

              yeah for me its mainly laziness so I don’t have to use an external site to search

              but new users who may not be aware of external directories could have trouble finding niche communities if no one from their instance has subscribed yet

              just a user experience thing I was thinking about