search engine indexing
4 votes c/dokk Posted by zPlus — 4 votes, 8 commentsSource

Anybody understands how google indexing work? Because I’ve noticed there’s an entire section of dokk that is not indexed, and I’m wondering if there’s something wrong with the website. Duckduckgo for comparison has a lot more indexed pages. The section is https://dokk.org/library, there are ~500 documents.

I only had a cursory look into SEO.

What I got from it was to add meta tags at least for title, author, description, and keywords. Dating the document by when it was published or updated also helps.

Next, is to be linked from a site a crawler actively revisits. I’m sure this would mean linking yourself on user-generated content hubs such as Reddit or Twitter because there’s always going to be new pages to discover from there, but Fediverse instances seldom if ever get placed on search engine indexes.

Thank you. “meta” do you mean literal HTML <meta> tags? Like <meta name="author">? I think these are supposed to refer to the HTML page itself rather than the content. For example “author” should mean who authored the HTML page, and not the author of the document that is displayed. If this makes sense.

Yeah
<title>…</title>
<meta name="description" content="..." />
<meta name="keywords" content="..." />
<meta name="author" content="..." />

I started adding those tags to my web pages to imrpove search rankings.

Also if you are writing an article, a search engine assigns more “weight” to an article that uses the keyword in a paragraph than just the heading. Like if you did:

\<h1>Keyword: My cool project this and that</h1>
\<p>My cool project works and it’s awesome.</p>

Your page would get ranked under anothe page that did:

\<h1>Keyword: Something else entirely with a similar name\</h1>
\<p>Keyword is a this-and-that under a completely different topic.\</p>

My understanding is that those tags help improve a page ranking. My problem is that the pages are not indexed in the first place, like at all. A search for site:dokk.org/library for example, or site:dokk.org/documentation, only returns 5 results (instead of thousands).

I am not sure that I understood you correctly but you are telling about search engines are lazy and don’t want to index pages on the site? If the answer is yes, don’t worry so much. First of all, dokk isn’t popular so the engines don’t have many reasons to index it, unfortunately. Secondly, itch is a very popular platform and page of the jams is also but even when my jam was on that page it didn’t appear in the google’s search output for week… Dokk isn’t so popular as itch so you can evaluate how much the engines wish to index the dokk. And I am not sure but does more ranking means the engine wants to index it more?

Sorry for the misunderstanding. I didn’t mean that they are lazy. I feel like there is some problem if they are not adding the pages to the index at all. Sometimes for example I’ve seen pages being added (I could search them) and then they were removed.

Regarding ranking and <meta> tags, what I meant is that these tags should only help a page with position. But if a page is not in the index at all, I don’t think these tags would make any difference.

By the way this only seems to be a problem with Google. If I search on other websites like Bing, DDG, or Yandex they are indexed.