open-source-mirrors
/
sherlock
mirror of https://github.com/sherlock-project/sherlock
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into restructure_take1

Browse Source
pull/350/head
Christopher K. Hoadley 5 years ago
parent 38b76486dd 376b3d8c69
commit 1587df0860
5 changed files with 450 additions and 302 deletions
Show Stats Download Patch File Download Diff File

26
README.md
Unescape View File

@ -36,7 +36,7 @@
## Demo ## Demo
You can use this link to test Sherlock directly in your browser: Use this link to test Sherlock directly in your browser:
https://elody.com/scenario/plan/16/ https://elody.com/scenario/plan/16/
## Installation ## Installation
@ -55,6 +55,7 @@ $ cd sherlock
# install the requirements # install the requirements
$ python3 -m pip install -r requirements.txt $ python3 -m pip install -r requirements.txt
``` ```
[![Open in Cloud Shell](https://gstatic.com/cloudssh/images/open-btn.png)](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/sherlock-project/sherlock&tutorial=README.md)
## Usage ## Usage
@ -68,7 +69,7 @@ usage: sherlock.py [-h] [--version] [--verbose] [--rank]
[--print-found] [--print-found]
USERNAMES [USERNAMES ...] USERNAMES [USERNAMES ...]
Sherlock: Find Usernames Across Social Networks (Version 0.9.1) Sherlock: Find Usernames Across Social Networks (Version 0.9.5)
positional arguments: positional arguments:
USERNAMES One or more usernames to check with social networks. USERNAMES One or more usernames to check with social networks.
@ -82,10 +83,10 @@ optional arguments:
rank in popularity. rank in popularity.
--folderoutput FOLDEROUTPUT, -fo FOLDEROUTPUT --folderoutput FOLDEROUTPUT, -fo FOLDEROUTPUT
If using multiple usernames, the output of the results If using multiple usernames, the output of the results
will be saved at this folder. will be saved to this folder.
--output OUTPUT, -o OUTPUT --output OUTPUT, -o OUTPUT
If using single username, the output of the result If using single username, the output of the result
will be saved at this file. will be saved to this file.
--tor, -t Make requests over Tor; increases runtime; requires --tor, -t Make requests over Tor; increases runtime; requires
Tor to be installed and in system path. Tor to be installed and in system path.
--unique-tor, -u Make requests over Tor with new Tor circuit after each --unique-tor, -u Make requests over Tor with new Tor circuit after each
@ -113,7 +114,7 @@ optional arguments:
``` ```
For example to search for only one user: To search for only one user:
``` ```
python3 sherlock.py user123 python3 sherlock.py user123
``` ```
@ -123,10 +124,13 @@ To search for more than one user:
python3 sherlock.py user1 user2 user3 python3 sherlock.py user1 user2 user3
``` ```
All of the accounts found will be stored in an individual text file with the corresponding username (e.g ```user123.txt```). Accounts found will be stored in an individual text file with the corresponding username (e.g ```user123.txt```).
## Anaconda (Windows) Notes
If you are using Anaconda in Windows, using 'python3' might not work. Use 'python' instead.
## Docker Notes ## Docker Notes
If you have docker installed you can build an image and run this as a container. If docker is installed you can build an image and run this as a container.
``` ```
docker build -t mysherlock-image . docker build -t mysherlock-image .
@ -142,7 +146,7 @@ The optional ```--rm``` flag removes the container filesystem on completion to p
The optional ```-t``` flag allocates a pseudo-TTY which allows colored output. See: https://docs.docker.com/engine/reference/run/#foreground The optional ```-t``` flag allocates a pseudo-TTY which allows colored output. See: https://docs.docker.com/engine/reference/run/#foreground
It is possible to use the following command to access the saved results: Use the following command to access the saved results:
``` ```
docker run --rm -t -v "$PWD/results:/opt/sherlock/results" mysherlock-image -o /opt/sherlock/results/text.txt user123 docker run --rm -t -v "$PWD/results:/opt/sherlock/results" mysherlock-image -o /opt/sherlock/results/text.txt user123
@ -152,14 +156,14 @@ The ```-v "$PWD/results:/opt/sherlock/results"``` option tells docker to create
present working directory and to mount it at `/opt/sherlock/results` on the docker container. present working directory and to mount it at `/opt/sherlock/results` on the docker container.
The `-o /opt/sherlock/results/text.txt` option tells `sherlock` to output the result. The `-o /opt/sherlock/results/text.txt` option tells `sherlock` to output the result.
Or you can simply use "Docker Hub" to run `sherlock`: Or you can use "Docker Hub" to run `sherlock`:
``` ```
docker run theyahya/sherlock user123 docker run theyahya/sherlock user123
``` ```
### Using `docker-compose` ### Using `docker-compose`
You can also use the `docker-compose.yml` file from the repository and use this command You can use the `docker-compose.yml` file from the repository and use this command:
``` ```
docker-compose run sherlok -o /opt/sherlock/results/text.txt user123 docker-compose run sherlok -o /opt/sherlock/results/text.txt user123
@ -174,7 +178,7 @@ to understand the issues.
**NOTE**: Sherlock is not accepting adult sites in the standard list. **NOTE**: Sherlock is not accepting adult sites in the standard list.
## Tests ## Tests
If you are contributing to Sherlock, then Thank You! Thank you for contributing to Sherlock!
Before creating a pull request with new development, please run the tests Before creating a pull request with new development, please run the tests
to ensure that everything is working great. It would also be a good idea to run the tests to ensure that everything is working great. It would also be a good idea to run the tests

2
docker-compose.yml
Unescape View File

@ -1,6 +1,6 @@
version: '2' version: '2'
services: services:
sherlok: sherlock:
build: . build: .
volumes: volumes:
- "./results:/opt/sherlock/results" - "./results:/opt/sherlock/results"

474
sherlock/resources/data.json
View File

File diff suppressed because it is too large Load Diff

22
sherlock/sherlock.py
Unescape View File

@ -26,7 +26,7 @@ from torrequest import TorRequest
from load_proxies import load_proxies_from_csv, check_proxy_list from load_proxies import load_proxies_from_csv, check_proxy_list
module_name = "Sherlock: Find Usernames Across Social Networks" module_name = "Sherlock: Find Usernames Across Social Networks"
__version__ = "0.9.1" __version__ = "0.9.5"
amount = 0 amount = 0
@ -149,7 +149,7 @@ def sherlock(username, site_data, verbose=False, tor=False, unique_tor=False, pr
proxy -- String indicating the proxy URL proxy -- String indicating the proxy URL
Return Value: Return Value:
Dictionary containing results from report. Key of dictionary is the name Dictionary containing results from report. Key of dictionary is the name
of the social network site, and the value is another dictionary with of the social network site, and the value is another dictionary with
the following keys: the following keys:
url_main: URL of main site. url_main: URL of main site.
@ -197,7 +197,7 @@ def sherlock(username, site_data, verbose=False, tor=False, unique_tor=False, pr
} }
if "headers" in net_info: if "headers" in net_info:
#Override/append any extra headers required by a given site. # Override/append any extra headers required by a given site.
headers.update(net_info["headers"]) headers.update(net_info["headers"])
# Don't make request if username is invalid for the site # Don't make request if username is invalid for the site
@ -216,11 +216,11 @@ def sherlock(username, site_data, verbose=False, tor=False, unique_tor=False, pr
results_site["url_user"] = url results_site["url_user"] = url
url_probe = net_info.get("urlProbe") url_probe = net_info.get("urlProbe")
if url_probe is None: if url_probe is None:
#Probe URL is normal one seen by people out on the web. # Probe URL is normal one seen by people out on the web.
url_probe = url url_probe = url
else: else:
#There is a special URL for probing existence separate # There is a special URL for probing existence separate
#from where the user profile normally can be found. # from where the user profile normally can be found.
url_probe = url_probe.format(username) url_probe = url_probe.format(username)
request_method = session.get request_method = session.get
@ -380,10 +380,10 @@ def main():
action="store_true", dest="rank", default=False, action="store_true", dest="rank", default=False,
help="Present websites ordered by their Alexa.com global rank in popularity.") help="Present websites ordered by their Alexa.com global rank in popularity.")
parser.add_argument("--folderoutput", "-fo", dest="folderoutput", parser.add_argument("--folderoutput", "-fo", dest="folderoutput",
help="If using multiple usernames, the output of the results will be saved at this folder." help="If using multiple usernames, the output of the results will be saved to this folder."
) )
parser.add_argument("--output", "-o", dest="output", parser.add_argument("--output", "-o", dest="output",
help="If using single username, the output of the result will be saved at this file." help="If using single username, the output of the result will be saved to this file."
) )
parser.add_argument("--tor", "-t", parser.add_argument("--tor", "-t",
action="store_true", dest="tor", default=False, action="store_true", dest="tor", default=False,
@ -398,7 +398,7 @@ def main():
parser.add_argument("--site", parser.add_argument("--site",
action="append", metavar='SITE_NAME', action="append", metavar='SITE_NAME',
dest="site_list", default=None, dest="site_list", default=None,
help="Limit analysis to just the listed sites. Add multiple options to specify more than one site." help="Limit analysis to just the listed sites. Add multiple options to specify more than one site."
) )
parser.add_argument("--proxy", "-p", metavar='PROXY_URL', parser.add_argument("--proxy", "-p", metavar='PROXY_URL',
action="store", dest="proxy", default=None, action="store", dest="proxy", default=None,
@ -464,7 +464,7 @@ def main():
else: else:
raise ValueError raise ValueError
except ValueError: except ValueError:
raise Exception("Prameter --check_proxies/-cp must be a positive intiger.") raise Exception("Parameter --check_proxies/-cp must be a positive integer.")
if args.tor or args.unique_tor: if args.tor or args.unique_tor:
print("Using Tor to make requests") print("Using Tor to make requests")
@ -505,7 +505,7 @@ def main():
if site_data_all is None: if site_data_all is None:
# Check if the file exists otherwise exit. # Check if the file exists otherwise exit.
if not os.path.exists(data_file_path): if not os.path.exists(data_file_path):
print("JSON file at doesn't exist.") print("JSON file doesn't exist.")
print( print(
"If this is not a file but a website, make sure you have appended http:// or https://.") "If this is not a file but a website, make sure you have appended http:// or https://.")
sys.exit(1) sys.exit(1)

228
sites.md
Unescape View File

@ -1,4 +1,4 @@
## List Of Supported Sites (176 Sites In Total!) ## List Of Supported Sites (186 Sites In Total!)
1. [2Dimensions](https://2Dimensions.com/) 1. [2Dimensions](https://2Dimensions.com/)
2. [500px](https://500px.com/) 2. [500px](https://500px.com/)
3. [9GAG](https://9gag.com/) 3. [9GAG](https://9gag.com/)
@ -68,112 +68,122 @@
67. [House-Mixes.com](https://www.house-mixes.com/) 67. [House-Mixes.com](https://www.house-mixes.com/)
68. [Houzz](https://houzz.com/) 68. [Houzz](https://houzz.com/)
69. [HubPages](https://hubpages.com/) 69. [HubPages](https://hubpages.com/)
70. [IFTTT](https://www.ifttt.com/) 70. [Hubski](https://hubski.com/)
71. [ImageShack](https://imageshack.us/) 71. [IFTTT](https://www.ifttt.com/)
72. [Instagram](https://www.instagram.com/) 72. [ImageShack](https://imageshack.us/)
73. [Instructables](https://www.instructables.com/) 73. [Instagram](https://www.instagram.com/)
74. [Investing.com](https://www.investing.com/) 74. [Instructables](https://www.instructables.com/)
75. [Issuu](https://issuu.com/) 75. [Investing.com](https://www.investing.com/)
76. [Itch.io](https://itch.io/) 76. [Issuu](https://issuu.com/)
77. [Jimdo](https://jimdosite.com/) 77. [Itch.io](https://itch.io/)
78. [Kaggle](https://www.kaggle.com/) 78. [Jimdo](https://jimdosite.com/)
79. [KanoWorld](https://world.kano.me/) 79. [Kaggle](https://www.kaggle.com/)
80. [Keybase](https://keybase.io/) 80. [KanoWorld](https://world.kano.me/)
81. [Kik](http://kik.me/) 81. [Keybase](https://keybase.io/)
82. [Kongregate](https://www.kongregate.com/) 82. [Kik](http://kik.me/)
83. [Launchpad](https://launchpad.net/) 83. [Kongregate](https://www.kongregate.com/)
84. [LeetCode](https://leetcode.com/) 84. [Launchpad](https://launchpad.net/)
85. [Letterboxd](https://letterboxd.com/) 85. [LeetCode](https://leetcode.com/)
86. [LiveJournal](https://www.livejournal.com/) 86. [Letterboxd](https://letterboxd.com/)
87. [Mastodon](https://mstdn.io/) 87. [LiveJournal](https://www.livejournal.com/)
88. [Medium](https://medium.com/) 88. [Lobsters](https://lobste.rs/)
89. [MeetMe](https://www.meetme.com/) 89. [Mastodon](https://mstdn.io/)
90. [MixCloud](https://www.mixcloud.com/) 90. [Medium](https://medium.com/)
91. [MyAnimeList](https://myanimelist.net/) 91. [MeetMe](https://www.meetme.com/)
92. [Myspace](https://myspace.com/) 92. [MixCloud](https://www.mixcloud.com/)
93. [NPM](https://www.npmjs.com/) 93. [MyAnimeList](https://myanimelist.net/)
94. [NPM-Package](https://www.npmjs.com/) 94. [Myspace](https://myspace.com/)
95. [NameMC (Minecraft.net skins)](https://namemc.com/) 95. [NPM](https://www.npmjs.com/)
96. [NationStates Nation](https://nationstates.net) 96. [NPM-Package](https://www.npmjs.com/)
97. [NationStates Region](https://nationstates.net) 97. [NameMC (Minecraft.net skins)](https://namemc.com/)
98. [Newgrounds](https://newgrounds.com) 98. [NationStates Nation](https://nationstates.net)
99. [OK](https://ok.ru/) 99. [NationStates Region](https://nationstates.net)
100. [OpenCollective](https://opencollective.com/) 100. [Newgrounds](https://newgrounds.com)
101. [Packagist](https://packagist.org/) 101. [OK](https://ok.ru/)
102. [Pastebin](https://pastebin.com/) 102. [OpenCollective](https://opencollective.com/)
103. [Patreon](https://www.patreon.com/) 103. [Packagist](https://packagist.org/)
104. [Pexels](https://www.pexels.com/) 104. [Pastebin](https://pastebin.com/)
105. [Photobucket](https://photobucket.com/) 105. [Patreon](https://www.patreon.com/)
106. [Pinterest](https://www.pinterest.com/) 106. [Pexels](https://www.pexels.com/)
107. [Pixabay](https://pixabay.com/) 107. [Photobucket](https://photobucket.com/)
108. [PlayStore](https://play.google.com/store) 108. [Pinterest](https://www.pinterest.com/)
109. [Plug.DJ](https://plug.dj/) 109. [Pixabay](https://pixabay.com/)
110. [Pokemon Showdown](https://pokemonshowdown.com) 110. [PlayStore](https://play.google.com/store)
111. [ProductHunt](https://www.producthunt.com/) 111. [Plug.DJ](https://plug.dj/)
112. [Quora](https://www.quora.com/) 112. [Pokemon Showdown](https://pokemonshowdown.com)
113. [Rajce.net](https://www.rajce.idnes.cz/) 113. [ProductHunt](https://www.producthunt.com/)
114. [Rate Your Music](https://rateyourmusic.com/) 114. [Quora](https://www.quora.com/)
115. [Reddit](https://www.reddit.com/) 115. [Rajce.net](https://www.rajce.idnes.cz/)
116. [Repl.it](https://repl.it/) 116. [Rate Your Music](https://rateyourmusic.com/)
117. [ResearchGate](https://www.researchgate.net/) 117. [Reddit](https://www.reddit.com/)
118. [ReverbNation](https://www.reverbnation.com/) 118. [Repl.it](https://repl.it/)
119. [Roblox](https://www.roblox.com/) 119. [ResearchGate](https://www.researchgate.net/)
120. [Scratch](https://scratch.mit.edu/) 120. [ReverbNation](https://www.reverbnation.com/)
121. [Scribd](https://www.scribd.com/) 121. [Roblox](https://www.roblox.com/)
122. [Signal](https://community.signalusers.org) 122. [Scratch](https://scratch.mit.edu/)
123. [Slack](https://slack.com) 123. [Scribd](https://www.scribd.com/)
124. [SlideShare](https://slideshare.net/) 124. [Signal](https://community.signalusers.org)
125. [Smashcast](https://www.smashcast.tv/) 125. [Slack](https://slack.com)
126. [SoundCloud](https://soundcloud.com/) 126. [SlideShare](https://slideshare.net/)
127. [SourceForge](https://sourceforge.net/) 127. [Smashcast](https://www.smashcast.tv/)
128. [Speedrun.com](https://speedrun.com/) 128. [SoundCloud](https://soundcloud.com/)
129. [Splits.io](https://splits.io) 129. [SourceForge](https://sourceforge.net/)
130. [Spotify](https://open.spotify.com/) 130. [Speedrun.com](https://speedrun.com/)
131. [Star Citizen](https://robertsspaceindustries.com/) 131. [Splits.io](https://splits.io)
132. [Steam](https://steamcommunity.com/) 132. [Spotify](https://open.spotify.com/)
133. [SteamGroup](https://steamcommunity.com/) 133. [Star Citizen](https://robertsspaceindustries.com/)
134. [T-MobileSupport](https://support.t-mobile.com) 134. [Steam](https://steamcommunity.com/)
135. [Taringa](https://taringa.net/) 135. [SteamGroup](https://steamcommunity.com/)
136. [Teknik](https://teknik.io/) 136. [T-MobileSupport](https://support.t-mobile.com)
137. [Telegram](https://t.me/) 137. [Taringa](https://taringa.net/)
138. [Tellonym.me](https://tellonym.me/) 138. [Teknik](https://teknik.io/)
139. [TikTok](https://www.tiktok.com/) 139. [Telegram](https://t.me/)
140. [Tinder](https://tinder.com/) 140. [Tellonym.me](https://tellonym.me/)
141. [TradingView](https://www.tradingview.com/) 141. [TikTok](https://www.tiktok.com/)
142. [Trakt](https://www.trakt.tv/) 142. [Tinder](https://tinder.com/)
143. [Trello](https://trello.com/) 143. [TradingView](https://www.tradingview.com/)
144. [Trip](https://www.trip.skyscanner.com/) 144. [Trakt](https://www.trakt.tv/)
145. [TripAdvisor](https://tripadvisor.com/) 145. [Trello](https://trello.com/)
146. [Twitch](https://www.twitch.tv/) 146. [Trip](https://www.trip.skyscanner.com/)
147. [Twitter](https://www.twitter.com/) 147. [TripAdvisor](https://tripadvisor.com/)
148. [Ultimate-Guitar](https://ultimate-guitar.com/) 148. [Twitch](https://www.twitch.tv/)
149. [Unsplash](https://unsplash.com/) 149. [Twitter](https://www.twitter.com/)
150. [VK](https://vk.com/) 150. [Ultimate-Guitar](https://ultimate-guitar.com/)
151. [VSCO](https://vsco.co/) 151. [Unsplash](https://unsplash.com/)
152. [Venmo](https://venmo.com/) 152. [VK](https://vk.com/)
153. [Vimeo](https://vimeo.com/) 153. [VSCO](https://vsco.co/)
154. [Virgool](https://virgool.io/) 154. [Venmo](https://venmo.com/)
155. [VirusTotal](https://www.virustotal.com/) 155. [Vimeo](https://vimeo.com/)
156. [Wattpad](https://www.wattpad.com/) 156. [Virgool](https://virgool.io/)
157. [We Heart It](https://weheartit.com/) 157. [VirusTotal](https://www.virustotal.com/)
158. [WebNode](https://www.webnode.cz/) 158. [Wattpad](https://www.wattpad.com/)
159. [Wikipedia](https://www.wikipedia.org/) 159. [We Heart It](https://weheartit.com/)
160. [Wix](https://wix.com/) 160. [WebNode](https://www.webnode.cz/)
161. [WordPress](https://wordpress.com) 161. [Wikipedia](https://www.wikipedia.org/)
162. [WordPressOrg](https://wordpress.org/) 162. [Wix](https://wix.com/)
163. [YouNow](https://www.younow.com/) 163. [WordPress](https://wordpress.com)
164. [YouPic](https://youpic.com/) 164. [WordPressOrg](https://wordpress.org/)
165. [YouTube](https://www.youtube.com/) 165. [YouNow](https://www.younow.com/)
166. [Zhihu](https://www.zhihu.com/) 166. [YouPic](https://youpic.com/)
167. [Zomato](https://www.zomato.com/) 167. [YouTube](https://www.youtube.com/)
168. [authorSTREAM](http://www.authorstream.com/) 168. [Zhihu](https://www.zhihu.com/)
169. [boingboing.net](https://boingboing.net/) 169. [Zomato](https://www.zomato.com/)
170. [devRant](https://devrant.com/) 170. [authorSTREAM](http://www.authorstream.com/)
171. [gfycat](https://gfycat.com/) 171. [boingboing.net](https://boingboing.net/)
172. [iMGSRC.RU](https://imgsrc.ru/) 172. [devRant](https://devrant.com/)
173. [last.fm](https://last.fm/) 173. [fanpop](http://www.fanpop.com/)
174. [mixer.com](https://mixer.com/) 174. [gfycat](https://gfycat.com/)
175. [osu!](https://osu.ppy.sh/) 175. [iMGSRC.RU](https://imgsrc.ru/)
176. [segmentfault](https://segmentfault.com/) 176. [last.fm](https://last.fm/)
177. [mixer.com](https://mixer.com/)
178. [osu!](https://osu.ppy.sh/)
179. [segmentfault](https://segmentfault.com/)
180. [Polygon](https://www.polygon.com/)
181. [Wikidot](http://www.wikidot.com/)
182. [GuruShots](https://gurushots.com/)
183. [Shockwave](http://www.shockwave.com/)
184. [ImgUp.cz](https://imgup.cz/)
185. [Alík.cz](https://www.alik.cz/)
186. [Sbazar.cz](https://www.sbazar.cz/)
Alexa.com rank data fetched at (2019-10-26 20:19:28.052980 UTC) Alexa.com rank data fetched at (2019-11-26 03:17:40.084724 UTC)

Write Preview
Loading…
Cancel
Save