NuntixBot
If you are reading this, NuntixBot probably turned up in your server logs and you want to know what it is. This page is the short answer.
NuntixBot collects material for nuntix.net, a bilingual technology news site. It is operated by the same person who publishes the site; contact details are in the imprint.
How to recognise it
NuntixBot/0.1 (+https://nuntix.net/en/bot.html)
That is the full User-Agent string. Anything else claiming to be NuntixBot
is not us — the string is trivial to forge and we have no way to stop that, so
verify by behaviour rather than by name if it matters to you.
What it fetches
Three things, and nothing else:
- RSS and Atom feeds of publications an editor has explicitly added as a source. This is the bulk of it.
- Article pages linked from those feeds, when the feed carries only a headline or a two-line teaser. One fetch per article, only when the feed entry is too thin to work with.
- Occasional pages found through a web search, when an editor asks for more coverage of a story that has only one source. At most five pages per request, and only when a person presses the button.
It does not crawl. There is no link-following, no site-walking, no discovery of pages beyond the three cases above.
How to stop it
Add this to your robots.txt:
User-agent: NuntixBot
Disallow: /
We fetch robots.txt before every request path listed above, cache it for six
hours per host, and honour what it says — including a Crawl-delay, when you
set one longer than our own. A Disallow naming NuntixBot takes effect within
six hours.
If your robots.txt is temporarily unreachable, we treat the site as open
rather than as forbidden, and note it in our log. If you would rather we
treated an outage as a refusal, write to us and we will.
If you want us gone faster than six hours, or gone entirely regardless of
robots.txt, the address in the imprint reaches a person.
How hard it pulls
- Conditional requests: we send
If-None-MatchandIf-Modified-Since, so an unchanged feed costs you a304and no body. - Polling interval adapts to how often a source actually publishes: five minutes at the fastest, six hours at the slowest.
- At least one second between requests, more if your
robots.txtasks. - Repeated failures widen the interval rather than retrying harder.
What we do with it
Articles on nuntix.net are written from several sources at once, with AI assistance, and checked by a human editor before publication. Every published article names the publications it was built from and links to them.
Three things we do not do, since they are the usual reasons to block a bot:
- We do not republish your text. Source material is internal working material. The database role that serves the public site is not granted access to the column holding it, so no public page can contain it even by mistake.
- We do not sell or share it, and it is not training data. It is used to write and fact-check one article, then it stays where it is.
- We are not an aggregator. No feed of your headlines, no framing of your pages. What gets published is original text that cites you.
More about how the site works is on the about page.