🍑 0.14.1 🍑

ass version 0.14.1
pull/197/head releases/0.14.1
Josh Moore 2 years ago committed by GitHub
commit e38adcaa28
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

40
.github/README.md vendored

@ -72,8 +72,7 @@ ass was designed with developers in mind. If you are a developer & want somethin
- Run locally or in a Docker container - Run locally or in a Docker container
- **Multiple file storage methods** - **Multiple file storage methods**
- Local file system - Local file system
- Amazon S3, including [DigitalOcean Spaces] - Amazon S3, including [DigitalOcean Spaces] (more coming soon)
- [Skynet] (free decentralized storage on the [Sia] blockchain)
- **Multiple data storage methods** using [data engines] - **Multiple data storage methods** using [data engines]
- **File** - **File**
- JSON (default, [papito]) - JSON (default, [papito])
@ -86,8 +85,6 @@ ass was designed with developers in mind. If you are a developer & want somethin
[Git Submodules]: https://git-scm.com/book/en/v2/Git-Tools-Submodules [Git Submodules]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
[ZWS]: https://zws.im [ZWS]: https://zws.im
[DigitalOcean Spaces]: https://www.digitalocean.com/products/spaces/ [DigitalOcean Spaces]: https://www.digitalocean.com/products/spaces/
[Skynet]: https://siasky.net/
[Sia]: https://sia.tech/
[data engines]: #data-engines [data engines]: #data-engines
[papito]: https://github.com/tycrek/papito [papito]: https://github.com/tycrek/papito
[ass-psql]: https://github.com/tycrek/ass-psql [ass-psql]: https://github.com/tycrek/ass-psql
@ -235,7 +232,9 @@ If you need to override a specific part of the config to be different from the g
| **`X-Ass-Domain`** | Override the domain returned for the clipboard (useful for multi-domain hosts) | | **`X-Ass-Domain`** | Override the domain returned for the clipboard (useful for multi-domain hosts) |
| **`X-Ass-Access`** | Override the generator used for the resource URL. Must be one of: `original`, `zws`, `gfycat`, `random`, or `timestamp` ([see above](#access-types)) | | **`X-Ass-Access`** | Override the generator used for the resource URL. Must be one of: `original`, `zws`, `gfycat`, `random`, or `timestamp` ([see above](#access-types)) |
| **`X-Ass-Gfycat`** | Override the length of Gfycat ID's. Defaults to `2` | | **`X-Ass-Gfycat`** | Override the length of Gfycat ID's. Defaults to `2` |
| **`X-Ass-Timeoffset`** | Override the timestamp offset. Defaults to `UTC+0` | | **`X-Ass-Timeoffset`** | Override the timestamp offset. Defaults to `UTC+0`. Available options are whatever [Luxon] accepts (for example: `America/Edmonton` or `UTC-7`) |
[Luxon]: https://moment.github.io/luxon/#/zones?id=specifying-a-zone
### Fancy embeds ### Fancy embeds
@ -324,6 +323,12 @@ By default, ass directs the index route `/` to this README. Follow these steps t
module.exports = (req, res, next) => res.redirect('/register'); module.exports = (req, res, next) => res.redirect('/register');
``` ```
## Custom 404 page
To use a custom 404 page, create a file in the `share/` directory called `404.html`. Restart ass, and any requests to missing resources will return HTTP 404 with the contents of this file.
If there's interest, I may allow making this a function, similar to the custom index.
## File storage ## File storage
ass supports three methods of file storage: local, S3, or [Skynet]. ass supports three methods of file storage: local, S3, or [Skynet].
@ -338,16 +343,6 @@ Any existing object storage server that's compatible with [Amazon S3] can be use
S3 servers are generally very fast & have very good uptime, though this will depend on the hosting provider & plan you choose. S3 servers are generally very fast & have very good uptime, though this will depend on the hosting provider & plan you choose.
### Skynet
**As of August 12, 2022, [Skynet Labs is shut down].** Skynet *will continue to work*, as such is the nature of decentralized services.
[Skynet] is a decentralized CDN created by [Skynet Labs]. It utilizes the [Sia] blockchain, the leading decentralized cloud storage platform, which boasts "no signups, no servers, no trusted third parties". For hosts who are looking for a reliable, always available storage solution with lots of capacity & no costs, Skynet may be your best option. However, uploads tend to be on the slower side (though speeds will improve as the Sia network grows).
[Skynet Labs is shut down]: https://skynetlabs.com/news/skynet-labs-shutting-down-skynet-remains-online
[Amazon S3]: https://en.wikipedia.org/wiki/Amazon_S3
[Skynet Labs]: https://github.com/SkynetLabs
## New user system (v0.14.0) ## New user system (v0.14.0)
The user system was overhauled in v0.14.0 to allow more features and flexibility. New fields on users include `admin`, `passhash`, `unid`, and `meta` (these will be documented more once the system is finalized). The user system was overhauled in v0.14.0 to allow more features and flexibility. New fields on users include `admin`, `passhash`, `unid`, and `meta` (these will be documented more once the system is finalized).
@ -388,18 +383,23 @@ Other things to note:
- **All endpoints are prefixed with `/api/`**. - **All endpoints are prefixed with `/api/`**.
- All endpoints will return a JSON object unless otherwise specified. - All endpoints will return a JSON object unless otherwise specified.
- Successful endpoints *should* return a `200` status code. Any errors will use the corresponding `4xx` or `5xx` status code (such as `401 Unauthorized`). - Successful endpoints *should* return a `200` status code. Any errors will use the corresponding `4xx` or `5xx` status code (such as `401 Unauthorized`).
- ass's API will try to be as compliant with the HTTP spec as possible. For example, using `POST/PUT` for create/modify, and response codes such as `409 Conflict` for duplicate entries. This compliance may not be 100% perfect, but I will try my best.
### API endpoints ### API endpoints
| Endpoint | Purpose | Admin? | | Endpoint | Purpose | Admin? |
| -------- | ------- | ------ | | -------- | ------- | ------ |
| **`GET /user/all`** | Returns a list of all users | Yes | | **`GET /user/`** | Returns a list of all users | Yes |
| **`GET /user/:id`** | Returns the user with the given ID | Yes |
| **`GET /user/self`** | Returns the current user | No | | **`GET /user/self`** | Returns the current user | No |
| **`GET /user/token/:token`** | Returns the user with the given token | No | | **`GET /user/token/:token`** | Returns the user with the given token | No |
| **`POST /user/reset`** | Resets the current user's **password** (token resets coming soon). Request body must be a JSON object including `username` and `password`. | No | | **`POST /user/`** | Creates a new user. Request body must be a JSON object including `username` and `password`. You may optionally include `admin` (boolean) or `meta` (object). Returns 400 if fails. | Yes |
| **`GET /user/:id`** | Returns the user with the given ID | Yes | | **`POST /user/password/reset/:id`** | Force resets the user's **password**. Request body must be a JSON object including a `password`. | Yes |
| **`POST /user/new`** | Creates a new user. Request body must be a JSON object including `username` and `password`. You may optionally include `admin` (boolean) or `meta` (object). Returns 400 if fails. | Yes | | **`DELETE /user/:id`** | Deletes the user with the given ID, as well as all their uploads. | Yes |
| **`PUT /user/meta/:id`** | Updates the user's metadata. Request body must be a JSON object with keys `key` and `value`, with the key/value you want to set in the users metadata. Optionally you may include `force: boolean` to override existing keys. | Yes |
| **`DELETE /user/meta/:id`** | Deletes a key/value from a users metadata. Request body must be a JSON object with a `key` property specifying the key to delete. | Yes |
| **`PUT /user/username/:id`** | Updates the user's username. Request body must be a JSON object with a `username` property. | Yes |
| **`PUT /user/token/:id`** | Regenerates a users upload token | Yes |
## Custom frontends - OUTDATED ## Custom frontends - OUTDATED

2629
package-lock.json generated

File diff suppressed because it is too large Load Diff

@ -1,6 +1,6 @@
{ {
"name": "ass", "name": "ass",
"version": "0.14.0-alpha.1", "version": "0.14.1",
"description": "The superior self-hosted ShareX server", "description": "The superior self-hosted ShareX server",
"main": "ass.js", "main": "ass.js",
"engines": { "engines": {
@ -42,7 +42,6 @@
"url": "https://patreon.com/tycrek" "url": "https://patreon.com/tycrek"
}, },
"dependencies": { "dependencies": {
"@skynetlabs/skynet-nodejs": "^2.3.0",
"@tsconfig/node16": "^1.0.1", "@tsconfig/node16": "^1.0.1",
"@tycrek/discord-hookr": "^0.1.0", "@tycrek/discord-hookr": "^0.1.0",
"@tycrek/express-nofavicon": "^1.0.3", "@tycrek/express-nofavicon": "^1.0.3",
@ -54,6 +53,7 @@
"any-shell-escape": "^0.1.1", "any-shell-escape": "^0.1.1",
"autoprefixer": "^10.4.4", "autoprefixer": "^10.4.4",
"aws-sdk": "^2.1115.0", "aws-sdk": "^2.1115.0",
"axios": "^1.2.1",
"bcrypt": "^5.1.0", "bcrypt": "^5.1.0",
"check-node-version": "^4.2.1", "check-node-version": "^4.2.1",
"crypto-random-string": "3.3.1", "crypto-random-string": "3.3.1",

@ -24,7 +24,7 @@ if (!fs.existsSync(configPath) || fs.readFileSync(configPath).toString().length
//#endregion //#endregion
// Load the JSON // Load the JSON
const { host, port, useSsl, isProxied, s3enabled, frontendName, useSia, diskFilePath }: Config = fs.readJsonSync(path('config.json')); const { host, port, useSsl, isProxied, s3enabled, frontendName, diskFilePath }: Config = fs.readJsonSync(path('config.json'));
const { CODE_INTERNAL_SERVER_ERROR }: MagicNumbers = fs.readJsonSync(path('MagicNumbers.json')); const { CODE_INTERNAL_SERVER_ERROR }: MagicNumbers = fs.readJsonSync(path('MagicNumbers.json'));
const { name, version, homepage }: Package = fs.readJsonSync(path('package.json')); const { name, version, homepage }: Package = fs.readJsonSync(path('package.json'));
@ -142,5 +142,5 @@ app.use((err: ErrWrap, _req: Request, res: Response) => log.error(err.message).e
.info('Frontend', ASS_FRONTEND.enabled ? ASS_FRONTEND.brand : 'disabled', `${ASS_FRONTEND.enabled ? `${getTrueHttp()}${getTrueDomain()}${ASS_FRONTEND.endpoint}` : ''}`) .info('Frontend', ASS_FRONTEND.enabled ? ASS_FRONTEND.brand : 'disabled', `${ASS_FRONTEND.enabled ? `${getTrueHttp()}${getTrueDomain()}${ASS_FRONTEND.endpoint}` : ''}`)
.info('Custom index', ASS_INDEX ?? 'disabled') .info('Custom index', ASS_INDEX ?? 'disabled')
.blank() .blank()
.express()!.Host(app, port, host, () => log.success('Ready for uploads', `Storing resources ${s3enabled ? 'in S3' : useSia ? 'on Sia blockchain' : 'on disk'}`)); .express()!.Host(app, port, host, () => log.success('Ready for uploads', `Storing resources ${s3enabled ? 'in S3' : 'on disk'}`));
})(); })();

@ -135,6 +135,12 @@ export const createNewUser = (username: string, password: string, admin: boolean
if (!authData.meta) authData.meta = {}; if (!authData.meta) authData.meta = {};
// Check if the CLI key is set
if (!authData.cliKey || authData.cliKey.length === 0) {
log.debug('CLI key is not set, generating new key');
authData.cliKey = nanoid(32);
}
fs.writeJson(authPath, authData, { spaces: '\t' }) fs.writeJson(authPath, authData, { spaces: '\t' })
.then(() => log.info('Created new user', newUser.username, newUser.unid)) .then(() => log.info('Created new user', newUser.username, newUser.unid))
.then(() => resolve(newUser)) .then(() => resolve(newUser))
@ -160,6 +166,168 @@ export const setUserPassword = (unid: string, password: string): Promise<User> =
const userIndex = authData.users.findIndex((user) => user.unid === unid); const userIndex = authData.users.findIndex((user) => user.unid === unid);
authData.users[userIndex] = user; authData.users[userIndex] = user;
fs.writeJson(authPath, authData, { spaces: '\t' }) fs.writeJson(authPath, authData, { spaces: '\t' })
.then(() => log.info('Set password for user', user.username, user.unid))
.then(() => resolve(user))
.catch(reject);
});
/**
* Deletes a user account
* @since v0.14.1
*/
export const deleteUser = (unid: string): Promise<void> => new Promise((resolve, reject) => {
// Find the user
const user = users.find((user) => user.unid === unid);
if (!user) return reject(new Error('User not found'));
// Remove the user from the users map
users.splice(users.indexOf(user), 1);
let fileCount: number;
// Remove the user's files
data().get().then((fileData: [string, FileData][]) => new Promise((resolve, reject) => {
// Create a queue of functions to run
const queue = fileData.map(([key, file]) => async () => {
if (file.uploader === unid) {
// Delete the file
const p = path(file.path);
await fs.unlink(p);
// Delete the thumbnail
const t = path(`uploads/thumbnails/${file.thumbnail}`);
await fs.unlink(t);
// Delete the info from the datafile
await data().del(key);
}
});
fileCount = queue.length;
// Recursively run the queue (see note above in `migrate()`)
const runQueue = (index: number) => {
if (index >= queue.length) return resolve(void 0);
queue[index]().then(() => runQueue(index + 1)).catch(reject);
};
runQueue(0);
})
.then(() => {
// Save the new user to auth.json
const authPath = path('auth.json');
const authData = fs.readJsonSync(authPath) as Users;
const userIndex = authData.users.findIndex((user) => user.unid === unid);
authData.users.splice(userIndex, 1);
return fs.writeJson(authPath, authData, { spaces: '\t' })
})
.then(() => log.info('Deleted user', user.unid, `${fileCount} files deleted`))
.then(() => resolve())
.catch(reject));
});
/**
* Sets a meta value for a user
* @since v0.14.1
*/
export const setUserMeta = (unid: string, key: string, value: any, force = false): Promise<User> => new Promise((resolve, reject) => {
// Find the user
const user = users.find((user) => user.unid === unid);
if (!user) return reject(new Error('User not found'));
// Set the meta value
if (user.meta[key] && !force) return reject(new Error('Meta key already exists'));
user.meta[key] = value;
// Save the new user to auth.json
const authPath = path('auth.json');
const authData = fs.readJsonSync(authPath) as Users;
const userIndex = authData.users.findIndex((user) => user.unid === unid);
authData.users[userIndex] = user;
fs.writeJson(authPath, authData, { spaces: '\t' })
.then(() => log.info('Set meta value for', user.unid, `${key}=${value}`))
.then(() => resolve(user))
.catch(reject);
});
/**
* Deletes a meta value for a user
* @since v0.14.1
*/
export const deleteUserMeta = (unid: string, key: string): Promise<User> => new Promise((resolve, reject) => {
// Find the user
const user = users.find((user) => user.unid === unid);
if (!user) return reject(new Error('User not found'));
// Delete the meta value
if (!user.meta[key]) return reject(new Error('Meta key does not exist'));
delete user.meta[key];
// Save the new user to auth.json
const authPath = path('auth.json');
const authData = fs.readJsonSync(authPath) as Users;
const userIndex = authData.users.findIndex((user) => user.unid === unid);
authData.users[userIndex] = user;
fs.writeJson(authPath, authData, { spaces: '\t' })
.then(() => log.info('Deleted meta value for', user.unid, key))
.then(() => resolve(user))
.catch(reject);
});
/**
* Sets the username for a user
* @since v0.14.1
*/
export const setUsername = (unid: string, username: string): Promise<User> => new Promise((resolve, reject) => {
// Find the user
const user = users.find((user) => user.unid === unid);
if (!user) return reject(new Error('User not found'));
// Check if the username is already taken
if (users.find((user) => user.username === username)) return reject(new Error('Username already taken'));
// Set the username
user.username = username;
// Save the new user to auth.json
const authPath = path('auth.json');
const authData = fs.readJsonSync(authPath) as Users;
const userIndex = authData.users.findIndex((user) => user.unid === unid);
authData.users[userIndex] = user;
fs.writeJson(authPath, authData, { spaces: '\t' })
.then(() => log.info('Set username for', user.unid, username))
.then(() => resolve(user))
.catch(reject);
});
/**
* Resets a token
* @since v0.14.1
*/
export const resetToken = (unid: string): Promise<User> => new Promise((resolve, reject) => {
// Find the user
const user = users.find((user) => user.unid === unid);
if (!user) return reject(new Error('User not found'));
// Reset the token
user.token = nanoid(32);
// Save the new user to auth.json
const authPath = path('auth.json');
const authData = fs.readJsonSync(authPath) as Users;
const userIndex = authData.users.findIndex((user) => user.unid === unid);
authData.users[userIndex] = user;
fs.writeJson(authPath, authData, { spaces: '\t' })
.then(() => log.info('Reset token for', user.unid))
.then(() => resolve(user)) .then(() => resolve(user))
.catch(reject); .catch(reject);
}); });
@ -204,13 +372,6 @@ export const onStart = (authFile = 'auth.json') => new Promise((resolve, reject)
return await createNewUser('ass', nanoid(), true); return await createNewUser('ass', nanoid(), true);
} }
// Check if the CLI key is set
if (!json.cliKey || json.cliKey.length === 0) {
log.debug('CLI key is not set, generating new key');
json.cliKey = nanoid(32);
fs.writeJsonSync(file, json, { spaces: '\t' });
}
// Add users to the map // Add users to the map
return json.users.forEach((user) => users.push(user)); return json.users.forEach((user) => users.push(user));
}) })

@ -5,7 +5,7 @@
*/ */
import { Router, Request, Response, NextFunction } from 'express'; import { Router, Request, Response, NextFunction } from 'express';
import { findFromToken, setUserPassword, users, createNewUser, verifyCliKey } from '../auth'; import { findFromToken, setUserPassword, users, createNewUser, deleteUser, setUserMeta, deleteUserMeta, setUsername, resetToken, verifyCliKey } from '../auth';
import { log } from '../utils'; import { log } from '../utils';
import { data } from '../data'; import { data } from '../data';
import { User } from '../types/auth'; import { User } from '../types/auth';
@ -15,6 +15,28 @@ import { User } from '../types/auth';
*/ */
const RouterApi = Router(); const RouterApi = Router();
/**
* Logs an error and sends a 500 (404 if 'User not found' error)
* @since v0.14.1
*/
const errorHandler = (res: Response, err: Error | any) => {
log.error(err);
// Get the status code for the Response
let code: number;
switch (err.message) {
case 'User not found':
code = 404; break;
case 'Meta key already exists':
case 'Username already taken':
code = 409; break;
default:
code = 500;
}
return res.status(code).type('text').send(err.message ?? err);
};
/** /**
* Token authentication middleware for Admins * Token authentication middleware for Admins
* @since v0.14.0 * @since v0.14.0
@ -33,12 +55,9 @@ const userFinder = (res: Response, user: User | undefined) => user ? res.json(us
function buildUserRouter() { function buildUserRouter() {
const userRouter = Router(); const userRouter = Router();
// Index // Index/Get all users
userRouter.get('/', (_req: Request, res: Response) => res.sendStatus(200));
// Get all users
// Admin only // Admin only
userRouter.get('/all', adminAuthMiddleware, (req: Request, res: Response) => res.json(users)); userRouter.get('/', adminAuthMiddleware, (req: Request, res: Response) => res.json(users));
// Get self // Get self
userRouter.get('/self', (req: Request, res: Response) => userRouter.get('/self', (req: Request, res: Response) =>
@ -50,18 +69,19 @@ function buildUserRouter() {
// Reset password (new plaintext password in form data; HOST SHOULD BE USING HTTPS) // Reset password (new plaintext password in form data; HOST SHOULD BE USING HTTPS)
// Admin only // Admin only
userRouter.post('/reset', adminAuthMiddleware, (req: Request, res: Response) => { // todo: user-resets using existing password
const id = req.body.id; userRouter.post('/password/reset/:id', adminAuthMiddleware, (req: Request, res: Response) => {
const id = req.params.id;
const newPassword = req.body.password; const newPassword = req.body.password;
setUserPassword(id, newPassword) setUserPassword(id, newPassword)
.then(() => res.sendStatus(200)) .then(() => res.sendStatus(200))
.catch((err) => (log.error(err), res.sendStatus(500))); .catch((err) => errorHandler(res, err));
}); });
// Create a new user // Create a new user
// Admin only // Admin only
userRouter.post('/new', adminAuthMiddleware, (req: Request, res: Response) => { userRouter.post('/', adminAuthMiddleware, (req: Request, res: Response) => {
const username: string | undefined = req.body.username; const username: string | undefined = req.body.username;
const password: string | undefined = req.body.password; const password: string | undefined = req.body.password;
const admin = req.body.admin ?? false; const admin = req.body.admin ?? false;
@ -73,7 +93,7 @@ function buildUserRouter() {
createNewUser(username, password, admin, meta) createNewUser(username, password, admin, meta)
.then((user) => res.send(user)) .then((user) => res.send(user))
.catch((err) => (log.error(err), res.sendStatus(500))); .catch((err) => errorHandler(res, err));
}); });
// Get a user (must be last as it's a catch-all) // Get a user (must be last as it's a catch-all)
@ -81,6 +101,72 @@ function buildUserRouter() {
userRouter.get('/:id', adminAuthMiddleware, (req: Request, res: Response) => userRouter.get('/:id', adminAuthMiddleware, (req: Request, res: Response) =>
userFinder(res, users.find(user => user.unid === req.params.id || user.username === req.params.id))); userFinder(res, users.find(user => user.unid === req.params.id || user.username === req.params.id)));
// Delete a user
// Admin only
userRouter.delete('/:id', adminAuthMiddleware, (req: Request, res: Response) => {
const id = req.params.id;
deleteUser(id)
.then(() => res.sendStatus(200))
.catch((err) => errorHandler(res, err));
});
// Update a user meta key/value (/meta can be after /:id because they are not HTTP GET)
// Admin only
userRouter.put('/meta/:id', adminAuthMiddleware, (req: Request, res: Response) => {
const id = req.params.id;
const key: string | undefined = req.body.key;
const value: any = req.body.value;
const force = req.body.force ?? false;
if (key == null || key.length === 0 || value == null || value.length === 0)
return res.sendStatus(400);
setUserMeta(id, key, value, force)
.then(() => res.sendStatus(200))
.catch((err) => errorHandler(res, err));
});
// Delete a user meta key
// Admin only
userRouter.delete('/meta/:id', adminAuthMiddleware, (req: Request, res: Response) => {
const id = req.params.id;
const key: string | undefined = req.body.key;
if (key == null || key.length === 0)
return res.sendStatus(400);
deleteUserMeta(id, key)
.then(() => res.sendStatus(200))
.catch((err) => errorHandler(res, err));
});
// Sets a username
// Admin only
// todo: allow users to change their own username
userRouter.put('/username/:id', adminAuthMiddleware, (req: Request, res: Response) => {
const id = req.params.id;
const username: string | undefined = req.body.username;
if (username == null || username.length === 0)
return res.sendStatus(400);
setUsername(id, username)
.then(() => res.sendStatus(200))
.catch((err) => errorHandler(res, err));
});
// Resets a token
// Admin only
// todo: allow users to reset their own token
userRouter.put('/token/:id', adminAuthMiddleware, (req: Request, res: Response) => {
const id = req.params.id;
resetToken(id)
.then(() => res.sendStatus(200))
.catch((err) => errorHandler(res, err));
});
return userRouter; return userRouter;
} }

@ -6,10 +6,9 @@ import escape from 'escape-html';
import fetch, { Response as FetchResponse } from 'node-fetch'; import fetch, { Response as FetchResponse } from 'node-fetch';
import { Request, Response } from 'express'; import { Request, Response } from 'express';
import { deleteS3 } from '../storage'; import { deleteS3 } from '../storage';
import { SkynetDelete, SkynetDownload } from '../skynet';
import { checkIfZws } from '../generators/zws'; import { checkIfZws } from '../generators/zws';
import { path, log, getTrueHttp, getTrueDomain, formatBytes, formatTimestamp, getS3url, getDirectUrl, getResourceColor, replaceholder } from '../utils'; import { path, log, getTrueHttp, getTrueDomain, formatBytes, formatTimestamp, getS3url, getDirectUrl, getResourceColor, replaceholder } from '../utils';
const { diskFilePath, s3enabled, viewDirect, useIdInViewer, idInViewerExtension, useSia }: Config = fs.readJsonSync(path('config.json')); const { diskFilePath, s3enabled, viewDirect, useIdInViewer, idInViewerExtension }: Config = fs.readJsonSync(path('config.json'));
const { CODE_UNAUTHORIZED, CODE_NOT_FOUND, }: MagicNumbers = fs.readJsonSync(path('MagicNumbers.json')); const { CODE_UNAUTHORIZED, CODE_NOT_FOUND, }: MagicNumbers = fs.readJsonSync(path('MagicNumbers.json'));
import { data } from '../data'; import { data } from '../data';
import { users } from '../auth'; import { users } from '../auth';
@ -22,14 +21,30 @@ if (fs.existsSync(path('share/', 'theme.json')))
theme = fs.readJsonSync(path('share/', 'theme.json')); theme = fs.readJsonSync(path('share/', 'theme.json'));
// Middleware for parsing the resource ID and handling 404 // Middleware for parsing the resource ID and handling 404
let custom404 = {
html: '',
checked: false,
error: null,
path: path('share/', '404.html')
};
router.use((req: Request, res: Response, next) => { router.use((req: Request, res: Response, next) => {
// Parse the resource ID // Parse the resource ID
req.ass = { resourceId: escape(req.resourceId || '').split('.')[0] }; req.ass = { resourceId: escape(req.resourceId || '').split('.')[0] };
// If the ID is invalid, return 404. Otherwise, continue normally // If the ID is invalid, return 404. Otherwise, continue normally
data().has(req.ass.resourceId) const processRequest = () => data().has(req.ass.resourceId)
.then((has: boolean) => has ? next() : res.sendStatus(CODE_NOT_FOUND)) // skipcq: JS-0229 .then((has: boolean) => has ? next() : custom404.html.length !== 0 ? res.status(CODE_NOT_FOUND).sendFile(custom404.path) : res.sendStatus(CODE_NOT_FOUND)) // skipcq: JS-0229
.catch(next); .catch(next);
// check if share/404.html exists
if (!custom404.checked)
fs.access(custom404.path, fs.constants.F_OK)
.then(() => fs.readFile(custom404.path, 'utf8'))
.then((data: string) => custom404.html = data)
.catch((err) => custom404.error = err)
.finally(() => (custom404.checked = true, log.debug('Custom 404', custom404.html.length !== 0 ? 'found' : 'not found', custom404.error ? `${custom404.error}` : 'no errors')))
.then(() => processRequest());
else processRequest();
}); });
// View file // View file
@ -80,9 +95,6 @@ router.get('/direct*', (req: Request, res: Response, next) => data().get(req.ass
file.headers.forEach((value, header) => res.setHeader(header, value)); file.headers.forEach((value, header) => res.setHeader(header, value));
file.body?.pipe(res); file.body?.pipe(res);
}), }),
sia: () => SkynetDownload(fileData)
.then((stream) => stream.pipe(res))
.then(() => SkynetDelete(fileData)),
local: () => fs.pathExists(path(fileData.path)) local: () => fs.pathExists(path(fileData.path))
.then((exists) => new Promise((resolve, reject) => !exists .then((exists) => new Promise((resolve, reject) => !exists
? reject(new Error('File does not exist')) ? reject(new Error('File does not exist'))
@ -92,7 +104,7 @@ router.get('/direct*', (req: Request, res: Response, next) => data().get(req.ass
.sendFile(path(fileData.path), (err) => err ? reject(err) : resolve(void 0)))) .sendFile(path(fileData.path), (err) => err ? reject(err) : resolve(void 0))))
}; };
return uploaders[fileData.randomId.startsWith('sia://') ? 'sia' : s3enabled ? 's3' : 'local'](); return uploaders[s3enabled ? 's3' : 'local']();
}).catch(next)); }).catch(next));
// Thumbnail response // Thumbnail response
@ -139,7 +151,7 @@ router.get('/delete/:deleteId', (req: Request, res: Response, next) => {
// Save the file information // Save the file information
return Promise.all([ return Promise.all([
s3enabled ? deleteS3(fileData) : !useSia ? fs.rmSync(path(fileData.path)) : () => Promise.resolve(), s3enabled ? deleteS3(fileData) : fs.rmSync(path(fileData.path)),
(!fileData.is || (fileData.is.image || fileData.is.video)) && fs.existsSync(path(diskFilePath, 'thumbnails/', fileData.thumbnail)) (!fileData.is || (fileData.is.image || fileData.is.video)) && fs.existsSync(path(diskFilePath, 'thumbnails/', fileData.thumbnail))
? fs.rmSync(path(diskFilePath, 'thumbnails/', fileData.thumbnail)) : () => Promise.resolve()]); ? fs.rmSync(path(diskFilePath, 'thumbnails/', fileData.thumbnail)) : () => Promise.resolve()]);
}) })

@ -16,7 +16,7 @@ const config = {
idInViewerExtension: false, idInViewerExtension: false,
dataEngine: '@tycrek/papito', dataEngine: '@tycrek/papito',
frontendName: 'ass-x', frontendName: 'ass-x',
useSia: false, savePerDay: false,
adminWebhookEnabled: false, adminWebhookEnabled: false,
s3enabled: false, s3enabled: false,
}; };
@ -46,6 +46,7 @@ const oldConfig = {
diskFilePath: 'uploads/', diskFilePath: 'uploads/',
saveWithDate: true, // Some systems don't like dirs with massive amounts of files saveWithDate: true, // Some systems don't like dirs with massive amounts of files
saveAsOriginal: false, // Prone to conflicts, which ass doesn't handle saveAsOriginal: false, // Prone to conflicts, which ass doesn't handle
useSia: false, // Sia has been shut down in 2022, uploads fail as of 2022-12-26
}; };
function getConfirmSchema(description) { function getConfirmSchema(description) {
@ -194,10 +195,10 @@ function doSetup() {
default: config.frontendName, default: config.frontendName,
required: false required: false
}, },
useSia: { savePerDay: {
description: 'Use Sia Skynet for decentralized file storage?', description: 'Save uploads in folders by day (YYYY-MM-DD) instead of by month (YYYY-MM)',
type: 'boolean', type: 'boolean',
default: config.useSia, default: config.savePerDay,
required: false required: false
}, },
adminWebhookEnabled: { adminWebhookEnabled: {

@ -1,37 +0,0 @@
import { FileData } from './types/definitions';
import fs, { ReadStream } from 'fs-extra';
import { path } from './utils';
import { SkynetClient } from '@skynetlabs/skynet-nodejs';
function getFullPath(fileData: FileData) {
return path('share', '.skynet', `${fileData.randomId}${fileData.ext}`.replace(/sia\:\/\//gi, ''));
}
// Create the SkyNet client
export const Skynet = new SkynetClient();
export function SkynetUpload(path: string): Promise<string> {
return new Promise(async (resolve, reject) => {
try {
const skylink = await Skynet.uploadFile(path);
resolve(skylink);
} catch (error) {
reject(error);
}
});
}
export function SkynetDownload(fileData: FileData): Promise<ReadStream> {
return new Promise((resolve: Function, reject) =>
fs.ensureDir(path('share', '.skynet'))
.then(async () => {
await Skynet.downloadFile(getFullPath(fileData), fileData.randomId);
return fs.createReadStream(getFullPath(fileData))
})
.then((stream) => resolve(stream))
.catch(reject));
}
export function SkynetDelete(fileData: FileData) {
return fs.remove(getFullPath(fileData));
}

@ -9,10 +9,9 @@ import Thumbnail from './thumbnails';
import Vibrant from './vibrant'; import Vibrant from './vibrant';
import Hash from './hash'; import Hash from './hash';
import { path, generateId, log } from './utils'; import { path, generateId, log } from './utils';
import { SkynetUpload } from './skynet';
import { Request, Response } from 'express'; import { Request, Response } from 'express';
import { removeGPS } from './nightmare'; import { removeGPS } from './nightmare';
const { s3enabled, s3endpoint, s3bucket, s3usePathStyle, s3accessKey, s3secretKey, diskFilePath, saveAsOriginal, saveWithDate, mediaStrict, maxUploadSize, useSia }: Config = fs.readJsonSync(path('config.json')); const { s3enabled, s3endpoint, s3bucket, s3usePathStyle, s3accessKey, s3secretKey, diskFilePath, saveAsOriginal, saveWithDate, savePerDay, mediaStrict, maxUploadSize }: Config = fs.readJsonSync(path('config.json'));
const { CODE_UNSUPPORTED_MEDIA_TYPE }: MagicNumbers = fs.readJsonSync(path('MagicNumbers.json')); const { CODE_UNSUPPORTED_MEDIA_TYPE }: MagicNumbers = fs.readJsonSync(path('MagicNumbers.json'));
const ID_GEN_LENGTH = 32; const ID_GEN_LENGTH = 32;
@ -34,8 +33,22 @@ function getDatedDirname() {
return `${diskFilePath}${diskFilePath.endsWith('/') ? '' : '/'}${year}-${`0${month}`.slice(-2)}`; // skipcq: JS-0074 return `${diskFilePath}${diskFilePath.endsWith('/') ? '' : '/'}${year}-${`0${month}`.slice(-2)}`; // skipcq: JS-0074
} }
/**
* A bit hacky but it works
* @since 0.14.1
*/
function getDatedDirnameWithDay() {
if (!savePerDay) return getDatedDirname();
// Get current day
const [, day] = new Date().toLocaleDateString('en-US').split('/');
// Add 0 before single digit days (6 turns into 06)
return `${getDatedDirname()}-${`0${day}`.slice(-2)}`; // skipcq: JS-0074
}
function getLocalFilename(req: Request) { function getLocalFilename(req: Request) {
let name = `${getDatedDirname()}/${saveAsOriginal ? req.file.originalname : req.file.sha1}`; let name = `${getDatedDirnameWithDay()}/${saveAsOriginal ? req.file.originalname : req.file.sha1}`;
// Append a number if this file has already been uploaded before // Append a number if this file has already been uploaded before
let count = 0; let count = 0;
@ -115,7 +128,7 @@ export function processUploaded(req: Request, res: Response, next: Function) { /
.catch((err) => log.debug('!! EXIF GPS data could not be removed', err)) .catch((err) => log.debug('!! EXIF GPS data could not be removed', err))
// Save file // Save file
.then(() => log.debug('Saving file', req.file.originalname, s3enabled ? 'in S3' : useSia ? 'on Sia blockchain' : 'on disk')) .then(() => log.debug('Saving file', req.file.originalname, s3enabled ? 'in S3' : 'on disk'))
.then(() => .then(() =>
// skipcq: JS-0229 // skipcq: JS-0229
new Promise((resolve, reject) => { new Promise((resolve, reject) => {
@ -129,18 +142,13 @@ export function processUploaded(req: Request, res: Response, next: Function) { /
Body: fs.createReadStream(req.file.path) Body: fs.createReadStream(req.file.path)
}).promise().then(resolve).catch(reject); }).promise().then(resolve).catch(reject);
// Use Sia Skynet
else if (useSia) return SkynetUpload(req.file.path)
.then((skylink) => req.file.randomId = skylink)
.then(resolve).catch(reject);
// Save to local storage // Save to local storage
else return fs.ensureDir(getDatedDirname()) else return fs.ensureDir(getDatedDirname())
.then(() => tempFileName = getLocalFilename(req)) .then(() => tempFileName = getLocalFilename(req))
.then(() => fs.copy(req.file.path, tempFileName, { preserveTimestamps: true })) .then(() => fs.copy(req.file.path, tempFileName, { preserveTimestamps: true }))
.then(resolve).catch(reject); .then(resolve).catch(reject);
})) }))
.then(() => log.debug('File saved', req.file.originalname, s3enabled ? 'in S3' : useSia ? 'on Sia blockchain' : 'on disk')) .then(() => log.debug('File saved', req.file.originalname, s3enabled ? 'in S3' : 'on disk'))
.catch((err) => next(err)) .catch((err) => next(err))
// Delete the file // Delete the file

@ -17,7 +17,7 @@ declare module 'ass-json' {
dataEngine: string dataEngine: string
frontendName: string frontendName: string
indexFile: string indexFile: string
useSia: boolean savePerDay: boolean
adminWebhookEnabled: boolean adminWebhookEnabled: boolean
adminWebhookUrl: string adminWebhookUrl: string
adminWebhookUsername: string adminWebhookUsername: string

Loading…
Cancel
Save