How can we improve Lemmy’s SEO so we can google “(question) lemmy” instead of relying on “(question) reddit” - eviltoast

most people i know use google by searching whatever question they have and including the word “reddit” at the end to find reddit threads since it currently has the most useful information.

As Lemmy gets more and more filled with useful threads and reviews it would be great if we can collectively improve Lemmy’s SEO so just including the word lemmy in a search will show lemmy threads related to the search.

The obscure tlds used in lemmy servers don’t help and lemmy.com currently redirects to lemm.ee. Is there a way we can improve the SEO of all instances or have lemmy.com be a aggregator of threads from many Lemmy servers?

  • WaterBottleOnAShelf@lemmy.nz
    link
    fedilink
    English
    arrow-up
    113
    arrow-down
    8
    ·
    1 year ago

    I think once there is enough info on lemmy it’ll just get to the point where the searches will bring up the information you need.

    I feel like if I’m searching on Google I want it to take me to the most relevant source of information even if that still is reddit. It won’t always be. But it is still right now.

    Nothing wrong with competition, it’ll give better search results.

  • empireOfLove@lemmy.one
    link
    fedilink
    English
    arrow-up
    77
    ·
    1 year ago

    Lemmy and other federated systems being spread out across individual instances does make things more difficult. Normally you could just do “site:reddit.com” and automatically filter all results to be from Reddit. But you can’t do that because your result could be on any one of hundreds of instances, many of which do not have “lemmy” in their title.

    It appear reasonably large instances like lemmy.one have been indexed, i got results using site:lemmy.one. and for those larger instances, they should still be able to index federated content.

    • DarraignTheSane@lemmy.world
      link
      fedilink
      English
      arrow-up
      12
      arrow-down
      1
      ·
      edit-2
      1 year ago

      Maybe I’m misinterpreting what you’re saying, but lemmy.one has basically no content on it other than the 8 communities that @jonah has created or allowed there. The whole point of that server is to allow people to simply login and then participate in other instances from there.

      That is all to say, lemmy.one would be one of the “smaller” instances from a standpoint of content to be indexed by Google.

      • Kichae@kbin.social
        link
        fedilink
        arrow-up
        26
        ·
        1 year ago

        The whole point of that server is to allow people to simply login and then participate in other instances from there.

        In order for users on lemmy.one to interact with content on other instances, lemmy.one has to import and host that content. So, it has plenty of content on it, just most of it originated elsewhere. That remote content should be just as indexable as local content.

        • Nat@apollo.town
          link
          fedilink
          arrow-up
          8
          ·
          edit-2
          1 year ago

          I think it would be best if each post had a canonical tag pointed at the originating server’s version of that post. The lemmy ui generates a canonical tag now but I’m not sure it doesn’t just point to itself.

        • DarraignTheSane@lemmy.world
          link
          fedilink
          arrow-up
          5
          ·
          1 year ago

          I didn’t quite realize that, figured that when you were viewing another instances’ content it was loading from that instance. I guess that means that Lemmy content across all instances has loads of redundant copies.

            • subnuggurat@lemmy.world
              link
              fedilink
              arrow-up
              2
              arrow-down
              1
              ·
              1 year ago

              Noob question here. Does that mean each instance must hold the totality of the content it knows about?

              • Kichae@kbin.social
                link
                fedilink
                arrow-up
                2
                ·
                1 year ago

                I’m not sure how lemmy or kbin handle instance-hosted media links – whether they import the media and redirect the link, or whether they point to the original media object – but otherwise, yes.

                There are ways to access other websites directly from within a given website – iframes and the like – but that’s not what happens here. Each website is independent of each other, and all text is locally hosted in your instance’s database.

                There are also (limited) copies of user profiles all over the place – if you click on my username, for instance, you’ll be taken to lemmy.world/u/Kichae@kbin.social. That’s a local lemmy.world user address, even though I’m not on lemmy.world. I can’t login to that account – it’s either credentialless, or has randomized credentials – but it exists. And by going there, you get to see what lemmy.world knows about my activity across the fediverse. Without ever leaving lemmy.world.

          • Tiritibambix@lemmy.ml
            link
            fedilink
            arrow-up
            9
            ·
            1 year ago

            No thanks. This adds absolutely no value to the conversation. I get the humor, but why would one want to turn Lemmy into reddit instead of going to reddit to sooth the “cravings” ?

            • dismalnow@kbin.social
              link
              fedilink
              arrow-up
              9
              arrow-down
              2
              ·
              edit-2
              1 year ago

              Eh… It has its place (especially if dude is going to set me up like that), and can be ignored if it’s not your cup of tea.

              Prurient banter has been a part of online forums since BBS message boards. It’s how most people I know communicate IRL and online: Bullshit, bullshit, nugget of wisdom, picture of cat.

              Not everything needs to be THUPER THERIAL, but it can be (and has frequently been) overdone because a lot of people don’t realize that self-censorship is a hammer to be swung heavily.

  • Reil@beehaw.org
    link
    fedilink
    English
    arrow-up
    50
    arrow-down
    1
    ·
    edit-2
    1 year ago

    It’ll happen if Lemmy gets big enough. I only worry about search engines getting tangled in the natural duplication of Lemmy posts.

    Like, if a web crawler sees a Beehaw post, and then seees Lemmy.ml’s mirrored page of that same post, could it just show up as two different results? Could it work against the SEO in that it gets marked as “duplicate” or “spam” content in some way?

    • dan@upvote.au
      link
      fedilink
      English
      arrow-up
      36
      ·
      1 year ago

      Like, if a web crawler sees a Beehaw post, and then seees Lemmy.ml’s mirrored page of that same post, could it just show up as two different results? Could it work against the SEO in that it gets marked as “duplicate” or “spam” content in some way?

      The ideal solution is that the page has a canonical tag, telling search engines what the main URL for the content is: https://ahrefs.com/blog/canonical-tags/. I don’t know if Lemmy already does this, nor do I know how well canonical tags work cross-domain as I’ve only ever used them for content on the same domain.

      • Olissipo@lemmy.pt
        link
        fedilink
        English
        arrow-up
        22
        ·
        1 year ago

        The ideal solution is that the page has a canonical tag, telling search engines what the main URL for the content is: https://ahrefs.com/blog/canonical-tags/. I don’t know if Lemmy already does this […]

        I checked and it does, this post’s canonical is:

        <link data-inferno-helmet="true" rel="canonical" href="https://merv.news/post/26663">

        Weirdly it uses OP’s instance, in this case merv.news. Shouldn’t it be the instance where it was posted?

        • AdmiralRob@lemmy.zip
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          I would think it’s because users only interact with their own instance. They would need to post it to their instance first before it can be forwarded to the appropriate community’s instance.

    • evatronic@lemm.ee
      link
      fedilink
      English
      arrow-up
      6
      ·
      1 year ago

      If/When Lemmy and other federated services grow to the point that’s an issue in major search engines, said search engines should be smart enough to group and/or suppress mirrored results.

      You can see that sort of thing in Google now for major sites like Reddit and StackOverflow, though it’s more along the lines of “the same question in a different post”.

      You can also, in the interim, just pick an instance and add, site:lemm.world or whatever instead of just “lemmy”.

    • ComradeKhoumrag@infosec.pub
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      It might help it, as well. I believe in the Yandex source code leak they detail their algorithms SEO techniques. Might be a good lead

  • hoodlem@hoodlem.me
    link
    fedilink
    English
    arrow-up
    51
    arrow-down
    7
    ·
    1 year ago

    Unpopular opinion, but I don’t want that. I don’t want to start adding SEO stuff. If we have good content Google can figure out how to index it better themselves.

    • Chozo@kbin.social
      link
      fedilink
      arrow-up
      28
      ·
      1 year ago

      Just curious, why not? Everything on the Fediverse is already public, by nature of federation. I think making the information shared here more easily discoverable is always a good thing.

        • Chozo@kbin.social
          link
          fedilink
          arrow-up
          13
          arrow-down
          1
          ·
          1 year ago

          SEO really doesn’t work organically, though. That’s why there’s a whole industry devoted to it.

          • koreth@lemm.ee
            link
            fedilink
            English
            arrow-up
            24
            ·
            1 year ago

            SEO is an industry devoted to undermining search engines’ ability to organically surface good content. Good content will still be surfaced on its own, just maybe not quite as quickly.

            • Chozo@kbin.social
              link
              fedilink
              arrow-up
              7
              ·
              edit-2
              1 year ago

              Good content will still be surfaced on its own

              Will it, though? This all seems like untested theory, to be honest.

              While SEO may have started as a means of manipulating search engines, search engines have grown to adopt to new SEO techniques and now use those techniques as part of their built-in ranking systems. Outside of content that goes truly “viral”, I think it’s pretty difficult to get anything new to the top of a Google search without some massive SEO these days. Especially considering the head start that bigger players have already gotten on their SEO game, and the sheer wealth of content that search engines have to parse through.

              I think maybe if we were still in 2010’s internet, that could be true. But search engines aren’t the same as they were in the past. SEO is the new norm.

              • wanderingmagus@lemmy.world
                link
                fedilink
                arrow-up
                5
                arrow-down
                1
                ·
                1 year ago

                What if we wanted to reject and change the new norm, just as we rejected Reddit and Twitter and started the Fediverse?

                • CascadianBeam@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  edit-2
                  1 year ago

                  The silent majority are never going to have the time to blow on shit that people who get so mentally invested in this stuff do.

                  That’s what people don’t seem to understand. The silent majority is not mindless by choice. They’re mindless by design.

              • Botree@lemmy.world
                link
                fedilink
                arrow-up
                1
                ·
                1 year ago

                I used to do a lot of SEO and run AdWords campaigns for smaller businesses back in the early days and they were always the norm. If anything Google has been constantly tweaking its algorithm to make it harder for non-organic SEO.

                Something as huge as Lemmy that grows organically doesn’t even have to worry about SEO. The problem is that the Fediverse being so spread out is a nightmare for Google spiders to crawl and rank.

                • Derproid@lemm.ee
                  link
                  fedilink
                  arrow-up
                  2
                  ·
                  1 year ago

                  The problem is that the Fediverse being so spread out is a nightmare for Google spiders to crawl and rank.

                  Isn’t that exactly why we should put in some basic SEO?

            • dan@upvote.au
              link
              fedilink
              English
              arrow-up
              3
              ·
              1 year ago

              Good content will still be surfaced on its own

              Not if you don’t do basic SEO at least… Things like ensuring that pages have:

              • Good title tags, with the most important parts (post topic) at the start of the title
              • Meta tags - description, keywords, canonical URL, etc
              • Open Graph tags - for when links are shared on social media sites
              • been optimized to load quickly - Google prioritises faster sites above slower ones

              Plus the site should have a robots.txt and sitemap XML that’s been submitted to Google Webmaster Tools.

          • liontigerwings@lemmy.world
            link
            fedilink
            arrow-up
            1
            ·
            1 year ago

            true when it comes to broad search terms, but google should bring lemmy to the top if you type in [search term] lemmy just like we do know with [search term] reddit.

      • slaytswiftfan@beehaw.org
        link
        fedilink
        arrow-up
        3
        ·
        1 year ago

        there’s a lot of “exclusivity” behaviour on this community that I’ve noticed (and a loooot of us vs the mentality too, in regards to twitter/reddit/bluesky/mastodon)

        • Chozo@kbin.social
          link
          fedilink
          arrow-up
          2
          ·
          1 year ago

          Yeah, I think a lot of the new Reddit refugees are failing to realize that Lemmy and the Fediverse existed before the recent migration and expect everything to just work the way they want, instead of how it’s been working just fine without them for years. The Fediverse doesn’t “belong” to them, and open connections are what this platform is built on.

    • APassenger@lemmy.one
      link
      fedilink
      English
      arrow-up
      7
      ·
      1 year ago

      When I read the posy what I heard was: how do we ensure high quality content? If we have that, SE won’t need the O.

      And I’m down for high quality.

  • KᑌᔕᕼIᗩ@lemmy.ml
    link
    fedilink
    English
    arrow-up
    42
    ·
    1 year ago

    If and when Lemmy proves it has staying power then search engines will adapt. It’s only been a month or so since it blew up.

  • Ab_intra@lemmy.world
    link
    fedilink
    English
    arrow-up
    36
    ·
    1 year ago

    SEO takes loads of time. And since every instance is it’s own thing it takes more times to propogate this. I can only see a future where the most popular instances and communities are searchable in search engines. But I hope I’m wrong.

  • AstralWeekends@lemm.ee
    link
    fedilink
    English
    arrow-up
    29
    ·
    1 year ago

    I think to some extent this is starting to work. I googled a software question the other day and found a lemmy answer within the first couple of results! Definitely better than a couple months ago where even searching for “lemmy” didn’t bring up lemmy among the top few.

    • Mike@lemmy.ml
      link
      fedilink
      English
      arrow-up
      16
      ·
      1 year ago

      Keep in mind how Google works. It’s taking your metrics into consideration. So someone who has never Googled lemmy likely wouldn’t have seen thst same result.

      • AstralWeekends@lemm.ee
        link
        fedilink
        English
        arrow-up
        5
        ·
        1 year ago

        That’s a great point, I don’t know much at all about SEO, but I do know how much Google tailors everything to your profile.

  • relative_iterator@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    20
    ·
    1 year ago

    Part of the problem is that there aren’t really a lot of questions/answers yet. You should try asking your questions here and hopefully over time we’ll have a big backlog of content.

  • candyman337@lemmy.world
    link
    fedilink
    English
    arrow-up
    18
    arrow-down
    1
    ·
    1 year ago

    I don’t think iw oild be unreasonable for someone to make a search engine that is specifically for indexing lemmy posts. Seems like it would be a good addition to the lemmyverse site

  • Tigbitties@kbin.social
    link
    fedilink
    arrow-up
    17
    ·
    1 year ago

    I’d rather find another solution. Imagine a fediverse search? Fuck google and their shitty SEO. It makes everything suck.

  • Spaceman Spiff@lemmy.fmhy.ml
    link
    fedilink
    English
    arrow-up
    14
    ·
    1 year ago

    While being decentralized certainly creates a barrier, most of the details behind PageRank (and the other algorithms in use by Google) are pretty well documented. If it doesn’t already, throwing in Lemmy as a keyword should soon bring up a Lemmy intense (probably Lemmy.ml or Lemmy.World) as a top result. As people click those links, the results will go higher.

    The bigger challenge is that the content you are trying to find isn’t here yet. Those results on the old site were built over years of massive user engagement. Lemmy has barely had a month since people started joining en masse, and it’s still a fraction of what we lost.

    TL;DR: Just keep using it and spread the word. The rest will happen naturally

  • HousePanther@lemmy.goblackcat.com
    link
    fedilink
    English
    arrow-up
    7
    arrow-down
    1
    ·
    1 year ago

    This is going to present a formidable challenge because Lemmy is so decentralized. I mean I am all for it but wouldn’t even begin to conceptualize how to get started.

    • AlternateRoute@lemmy.ca
      link
      fedilink
      English
      arrow-up
      9
      ·
      1 year ago

      Not really… Nearly every phpBB, SMF, Invision forum etc is a separate self hosted community and they get indexed. Hell even small ones like one of my gaming guilds ranked high for specific key words.

      For SEO to work well you need clear links to clear topics, and it helps to have automatic metada also to assist indexing.

      Having google compatible site maps assists in crawling the site.

      • ryathal@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        They can all get crawled, but you can’t just add lemmy and force results from the fediverse to the top.

        • AlternateRoute@lemmy.ca
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          You can still do SEO (search engine optimization) but making sure all the prefered metadata is there, sitemaps etc… It improves your rank.

          Isn’t going to put you at the top, that is the job of the content but it can be the difference between being on page one or not.