ffa2310630
Add LinkedIn
...
LinkedIn requires users to be signed in for normal visitation, but certain high-trust crawlers are allowed as well. Here, we masquerade as Googlebot to get past the 999.
8 months ago
198798d28b
Add basic schema
8 months ago
e2faa8a281
remove default username_unclaimed
8 months ago
f224cbe656
add AniWorld
8 months ago
49b190f841
Updated Site List
8 months ago
743aa8d320
Merge pull request #2097 from sherlock-project/hudsonrock
...
added HudsonRock
8 months ago
f035cc329d
added HudsonRock
8 months ago
58a2cc9c43
Updated Site List
8 months ago
667289a23a
Merge pull request #2092 from ppfeister/feature/new_targets
...
Collection of new and restored targets
8 months ago
a7f94d6ee6
Merge branch 'master' into feature/new_targets
8 months ago
122082a602
Merge pull request #2069 from ppfeister/feature/antiwaf
...
Reduce false positives (those caused by WAFs and bot detection)
8 months ago
71bdf63c19
Updated Site List
8 months ago
482982c4e4
Merge pull request #2068 from ppfeister/master
...
Reduce false positives (those not caused by WAFs or bot detection)
8 months ago
9137ee4f09
Add ruff to pr tests ( #2093 )
9 months ago
2bf6ee4d2e
Fix 1337x
9 months ago
7bf4be4b70
Add Untappd
...
Untappd is behind Cloudflare, but this doesn't seem to impact early requests. After being ratelimited, sherlock-project/sherlock#2069 will suppress any possible WAF-induced false positives.
Closes sherlock-project/sherlock#1780
9 months ago
760bc98a45
Re-add Instagram
...
Instagram has been added back to the site list using the old picuki probe. This service DOES use Cloudflare, but it will function perfectly fine until rate limited. Once rate limited, sherlock-project/sherlock#2069 will suppress any false positives.
9 months ago
89251745d9
Add Codeberg
9 months ago
db213de326
Remove unnecessary errorMsg
9 months ago
7c94534732
Add CSSBattle ~properly~
9 months ago
137009ef37
Add Spells8 (and sort)
...
Resolves sherlock-project/sherlock#2091
9 months ago
51959f95a7
Update sites list
9 months ago
2a5fbd9e97
Remove BitcoinForum instead of suppress
9 months ago
ca9a78757f
Add CyberDefenders
9 months ago
5c23b1555c
Update default User-Agent
...
I've noticed that many bot-detection pages are able to be avoided by using this UA. Unless there's a reason to stay on the old old old one, we may as well update it and reduce our WAF hits.
9 months ago
9f6cd80e28
Add PentesterLab
9 months ago
27e98abbc8
Add some new targets
...
+ TorrentGalaxy
+ 1337x
+ HackenProof
9 months ago
d59661903a
Update site lists
9 months ago
e43c6047be
Fix status_code default logic bug
9 months ago
060088f7a1
Fix status_code logic bug
9 months ago
da3c90fce8
Fix Telegram F+
9 months ago
482f216ff9
Fix Kongregate F+
...
Attempts were met with a Varnish error page presenting 54113 (possibly Fastly related).
Change to User Agent necessary to avoid Varnish/Fastly issues.
Change to Accept necessary to avoid infinite 302 redirection.
Without BOTH of these changes, attempts will fail.
Both changes being made also permit the use of status_code rather than message.
9 months ago
4006556daa
Remove G2G for F+
9 months ago
48212f09be
Fix adtl Genius A/U F+
9 months ago
936c507bfb
Fix jeuxvideo 403 F+
9 months ago
a56b301e3f
Fix GeeksForGeeks F+
9 months ago
c4777afd7f
Fix . F+ epidemic
9 months ago
d5c235df71
Fix Minecraft rate limit F+
9 months ago
cc95d4eea9
Fix HackerNews rate limit F+
9 months ago
5d5d8075b9
Revert from ? to -
9 months ago
b00474a2bf
Add PerimeterX fingerprint
9 months ago
1b14b88043
Fix Yandex Captcha F+
9 months ago
b6564a8527
Even more specific CNET
9 months ago
e6d0f1ee7b
Fix Slides F+, Add multi err code support
...
Error codes module expanded to support arrays of error codes rather than only one.
Using this new functionality, Slides was set to error codes 404 (as standard) AND 204 (non standard), to accomodate for that website's odd edge case.
9 months ago
eb89787dcd
Fix CNET F+ for bad unames
9 months ago
ca1d6f1418
Fix Telegram F+/F-
9 months ago
2ba87e23a7
Fix Genius[.]com Artists
9 months ago
8a05ce0817
Update CNET (again)
9 months ago
d660c1a2dd
Filter WAF Hits
9 months ago
0b610b49ae
Update removed sites list
9 months ago
Paul Pfeister
Ableytner
github-actions[bot]
Siddharth Dushantha
Ryan