From 379725e796bc6c1e4601f9524c85cbc9ec804b64 Mon Sep 17 00:00:00 2001 From: Paul Pfeister Date: Wed, 15 May 2024 15:20:26 -0400 Subject: [PATCH] Refresh documentation --- CONTRIBUTING.md | 31 ----- CODE_OF_CONDUCT.md => docs/CODE_OF_CONDUCT.md | 0 docs/CONTRIBUTING.md | 88 +++++++++++++ README.md => docs/README.md | 124 +++++++----------- docs/install.md | 95 ++++++++++++++ 5 files changed, 230 insertions(+), 108 deletions(-) delete mode 100644 CONTRIBUTING.md rename CODE_OF_CONDUCT.md => docs/CODE_OF_CONDUCT.md (100%) create mode 100644 docs/CONTRIBUTING.md rename README.md => docs/README.md (60%) create mode 100644 docs/install.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 7a2fa3fe..00000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,31 +0,0 @@ -# How To Contribute To Sherlock -First off, thank you for the help! - -There are many ways to contribute. Here is some high level grouping. - -## Adding New Sites - -Please look at the Wiki entry on -[adding new sites](https://github.com/sherlock-project/sherlock/wiki/Adding-Sites-To-Sherlock) -to understand the issues. - -Any new sites that are added need to have a username that has been claimed, and one -that is unclaimed documented in the site data. This allows the regression tests -to ensure that everything is working. - -It is required that a contributor test any new sites by either running the full tests, or running -a site-specific query against the claimed and unclaimed usernames. - -It is not required that a contributor run the -[site_list.py](https://github.com/sherlock-project/sherlock/blob/master/site_list.py) -script. - -If there are performance problems with a site (e.g. slow to respond, unreliable uptime, ...), then -the site may be removed from the list. The -[removed_sites.md](https://github.com/sherlock-project/sherlock/blob/master/removed_sites.md) -file contains sites that were included at one time in Sherlock, but had to be removed for -one reason or another. - -## Adding New Functionality - -Please ensure that the content on your branch passes all tests before submitting a pull request. diff --git a/CODE_OF_CONDUCT.md b/docs/CODE_OF_CONDUCT.md similarity index 100% rename from CODE_OF_CONDUCT.md rename to docs/CODE_OF_CONDUCT.md diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 00000000..4f1ff4c3 --- /dev/null +++ b/docs/CONTRIBUTING.md @@ -0,0 +1,88 @@ +

+
+ +
+

+ +

+ Home +    |    + Installation +    |    + Usage +    |    + Docker +    |    + Contributing +

+ +# How to contribute to Sherlock + +We would love to have you help us with the development of Sherlock. Each and every contribution is greatly valued! + +Here are some things we would appreciate your help on: +1. [Adding targets](#adding-targets) +1. [Cleaning up existing targets](#removing-targets) +1. [Restoring previously removed targets](#restoring-targets) + +## Adding targets + +Please look at the Wiki entry on [adding new sites][wiki_new_sites] to understand the issues. + +All new sites that are added to Sherlock need to have an existing (already claimed) username included in their definition. The linked Wiki page describes this in more detail. This inclusion allows us to run unit tests and prevent regression. + +Contributors are *required* to test any new sites for both false positives and false negatives. Contributors are *encouraged* to run unit tests as well. + +Contributors do not have to run the [site_list.py](/site_list.py) script, as it's ran automagically on master after each manifest change. + +## Removing targets + +If there are performance problems with a site (e.g. slow to respond, unreliable uptime, ...), then +the site may be removed from the list. The [removed_sites.md][file_removed_md] file contains sites that were included at one time in Sherlock, but had to be removed for one reason or another. + +If a site has *occasional* performance problems, but is otherwise accurate, it may be preferable to add a test to weed out false positives rather than removing it. + +## Restoring targets + +Likely our biggest backlog. If you can propose a functional query that complies with [#Adding targets](#adding-targets) that would shrink our [removed sites list][file_removed_md], that would be greatly appreciated. + +## Adding New Functionality + +Contributors that would like to add a feature to Sherlock should open an new [issue][issues_new], proposing their idea. Indicate that you would like to make a Pull Request for said feature. + +Creating an Issue prior to opening a PR helps with tracking, discussions, and avoids hurt feelings if for whatever reason we don't feel that a feature is compatible with the project. + +Please ensure that the content on your branch passes all tests before submitting a pull request. + +# Coverage and Unit Tests + +Thank you for contributing to Sherlock! + +Before creating a pull request with new development, please run the tests +to ensure that everything is working great. It would also be a good idea to run the tests +before starting development to distinguish problems between your +environment and the Sherlock software. + +The following is an example of the command line to run all the tests for +Sherlock. This invocation hides the progress text that Sherlock normally +outputs, and instead shows the verbose output of the tests. + +```console +$ cd sherlock/sherlock +$ python3 -m unittest tests.all --verbose +``` + +Unfortunately, some of the sites that Sherlock checks are not always reliable, so it is common +to get response problems. Any problems in connection will show up as warnings in the tests instead of true errors. + +If some sites are failing due to connection problems (site is down, in maintenance, etc) you can exclude them from tests by creating a `tests/.excluded_sites` file with a list of sites to ignore (one site name per line). + +## Coverage for new features + +Contributors that add new features are *encouraged* make an attempt at creating unit tests for them, as well. Not all contributions are suitable for unit tests, but when it's doable, it helps prevent regression. + + + +[wiki_new_sites]: https://github.com/sherlock-project/sherlock/wiki/Adding-Sites-To-Sherlock +[file_removed_md]: /removed_sites.md +[issues_new]: https://github.com/sherlock-project/sherlock/issues/new/choose \ No newline at end of file diff --git a/README.md b/docs/README.md similarity index 60% rename from README.md rename to docs/README.md index b4783448..fe3bcafc 100644 --- a/README.md +++ b/docs/README.md @@ -2,18 +2,18 @@

- Hunt down social media accounts by username across social networks + Hunt down social media accounts by username across 400+ social networks

- Installation + Installation    |    - Usage + Usage    |    - Docker Notes + Docker    |    - Contributing + Contributing

@@ -24,9 +24,22 @@ ## Installation -```console -pip install git+https://github.com/sherlock-project/sherlock.git -``` +[![PyPI - Version](https://img.shields.io/pypi/v/sherlock-project?logo=PyPi&label=PyPI&color=darkgreen)][ext_pypi] [![Docker Image Version](https://img.shields.io/docker/v/sherlock/sherlock?sort=semver&logo=docker&label=Docker&color=darkgreen)][docs_docker] [![homebrew version](https://img.shields.io/homebrew/v/sherlock?logo=Homebrew&color=darkgreen)][ext_brew] + + +| Method | Command | Notes | +| - | - | - | +| pypi | `pipx install sherlock-project` | `pip` may be used in place of `pipx` | +| brew | `brew install sherlock` | Community supported | +| docker | `docker pull sherlock/sherlock` | | + +### Alternative guides and methods + +- [See all alternative guides][docs_install] +- [Python package][docs_py] +- [Docker container][docs_docker] + + ## Usage @@ -80,88 +93,28 @@ optional arguments: ``` To search for only one user: -``` -python3 sherlock user123 +```bash +sherlock user123 ``` To search for more than one user: -``` -python3 sherlock user1 user2 user3 +```bash +sherlock user1 user2 user3 ``` Accounts found will be stored in an individual text file with the corresponding username (e.g ```user123.txt```). -## Anaconda (Windows) Notes - -If you are using Anaconda in Windows, using `python3` might not work. Use `python` instead. -## Docker Notes +## How to contribute to Sherlock -If docker is installed you can build an image and run this as a container. - -``` -docker build -t mysherlock-image . -``` - -Once the image is built, sherlock can be invoked by running the following: - -``` -docker run --rm -t mysherlock-image user123 -``` - -Use the following command to access the saved results: - -``` -docker run --rm -t -v "$PWD/results:/opt/sherlock/results" mysherlock-image -o /opt/sherlock/results/text.txt user123 -``` - -Docker is instructed to create (or use) the folder `results` in the current working directory and to mount it at `/opt/sherlock/results` on the docker container by using the ```-v "$PWD/results:/opt/sherlock/results"``` options. `Sherlock` is instructed to export the result using the `-o /opt/sherlock/results/text.txt` option. - - -### Using `docker-compose` - -You can use the `docker-compose.yml` file from the repository and use this command: - -``` -docker-compose run sherlock -o /opt/sherlock/results/text.txt user123 -``` - -## Contributing We would love to have you help us with the development of Sherlock. Each and every contribution is greatly valued! Here are some things we would appreciate your help on: -- Addition of new site support ¹ -- Bringing back site support of [sites that have been removed](removed_sites.md) in the past due to false positives - -[1] Please look at the Wiki entry on [adding new sites](https://github.com/sherlock-project/sherlock/wiki/Adding-Sites-To-Sherlock) -to understand the issues. +- [Adding targets][docs_contrib_adding_targets] +- [Cleaning up existing targets][docs_contrib_removing_targets] +- [Restoring previously removed targets][docs_contrib_restoring_targets] -## Tests - -Thank you for contributing to Sherlock! - -Before creating a pull request with new development, please run the tests -to ensure that everything is working great. It would also be a good idea to run the tests -before starting development to distinguish problems between your -environment and the Sherlock software. - -The following is an example of the command line to run all the tests for -Sherlock. This invocation hides the progress text that Sherlock normally -outputs, and instead shows the verbose output of the tests. - -```console -$ cd sherlock/sherlock -$ python3 -m unittest tests.all --verbose -``` - -Note that we do currently have 100% test coverage. Unfortunately, some of -the sites that Sherlock checks are not always reliable, so it is common -to get response problems. Any problems in connection will show up as -warnings in the tests instead of true errors. - -If some sites are failing due to connection problems (site is down, in maintenance, etc) -you can exclude them from tests by creating a `tests/.excluded_sites` file with a -list of sites to ignore (one site name per line). +Head over to our __[Contributing][docs_contrib]__ page for additional info. ## Star History @@ -175,3 +128,20 @@ list of sites to ignore (one site name per line). MIT © Sherlock Project
Original Creator - [Siddharth Dushantha](https://github.com/sdushantha) + + + +[docs_install]: /docs/install.md +[docs_docker]: /docs/install.md#docker +[docs_docker_dockerhub]: /docs/install.md#docker +[docs_docker_compose]: /docs/install.md#using-compose +[docs_docker_source]: /docs/install.md#build-image-from-source-useful-for-contributors +[docs_py]: /docs/install.md#python +[docs_py_build]: /docs/install.md#build-python-package-from-source-useful-for-contributors +[docs_contrib]: /docs/CONTRIBUTING.md +[docs_contrib_adding_targets]: /docs/CONTRIBUTING.md#adding-targets +[docs_contrib_removing_targets]: /docs/CONTRIBUTING.md#removing-targets +[docs_contrib_restoring_targets]: /docs/CONTRIBUTING.md#restoring-targets +[ext_pypi]: https://pypi.org/project/sherlock-project/ +[ext_brew]: https://formulae.brew.sh/formula/sherlock + diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 00000000..9813a09c --- /dev/null +++ b/docs/install.md @@ -0,0 +1,95 @@ +

+
+ +
+

+ +

+ Home +    |    + Installation +    |    + Usage +    |    + Docker +    |    + Contributing +

+ + + +# Alternative install methods + +1. __[Python package](#python)__ + 1. [Build from source](#build-and-install-package-from-source) +1. __[Docker Container](#docker)__ + 1. [via Docker Hub (one off)](#docker) + 1. [via Docker Hub (compose)](#using-compose) + 1. [Local image with Dockerfile](#build-image-from-source-alternative-to-docker-hub) + +

+Python +PyPI - Version +

+ +```bash +# pipx is recommended, but pip may suffice if pipx is unavailable +pipx install sherlock-project +``` + +### Build python package from source (useful for contributors) + +```bash +# pipx is recommended, but pip may suffice if pipx is unavailable +git clone https://github.com/sherlock-project/sherlock.git +cd sherlock +pipx install . +``` + +

+Docker +Docker Image Version +

+ +> [!NOTE] +> Sherlock doesn't yet have context detection. It's recommended that Docker containers be ran with option `-o /opt/sherlock/results/{user123}.txt` (replace {user123}) when an output file is desired at the mounted volume (as seen in the compose). +> +> This has no effect on stdout, which functions as expected out of the box. + +```bash +# One-off searches +docker run --rm -t sherlock/sherlock user123 + +# If you need to save the output file... (modify as needed) +# Output file will land in ${pwd}/results +docker run --rm -t -v "$PWD/results:/opt/sherlock/results" sherlock/sherlock -o /opt/sherlock/results/text.txt user123 +``` + +```bash +# At any time, you may update the image via this command +docker pull sherlock/sherlock +``` + +### Using compose + +```yml +version: "3" +services: + sherlock: + container_name: sherlock + image: sherlock/sherlock + volumes: + - ./sherlock/:/opt/sherlock/results/ +``` + +```bash +docker compose run sherlock user123 +``` + +### Build image from source (useful for contributors) + +```bash +# Assumes ${pwd} is repository root +docker build -t sherlock . +docker run --rm -t sherlock user123 +```