A new `log_janitor` setting added to `settings.yml` to allow the user to
specify the maximum number of files to keep when doing log cleanup.
Additionally, the way startup tasks occur has been cleaned up /
refactored to make it easier to integration test.
Simplify the factory pattern so that it creates settings more directly
instead of using a silly setter method.
Also implemented `IntegrationFixture` for easier integration testing.
In order to improve the stability and predictability of releases, I'm
favoring fixed version numbers over wildcards. This means upgrades will
be more tedious, but at least it will be easier to audit when things
break due to third party libraries.
Initialization logic has been completely overhauled. The previous
implementation was based on an approach that prioritized keeping the
composition root in the Program class. However, I wasn't happy with
this. CliFx inevitably wants to be the effective entry point to the
application. This means that the Program class should be as dumb as
possible.
The motivation for all this rework is the Recyclarr GUI. I need to be
able to share more initialization code between the projects.
Along with the initialization logic changes, I unintentionally
interleaved in another, completely unrelated refactoring. The IAppPaths
class now uses `IFileInfo` / `IDirectoryInfo` instead of `string` for
everything. This greatly simplified the implementation of that interface
and reduced dependencies and complexity across the code base. However,
those changes were vast and required rewriting/fixing a lot of unit
tests.
This sets the groundwork for making Radarr guide data available for
other usages beyond syncing to Radarr, such as spitting out information
to the console.
When you specify an empty object in YAML, like:
```
quality_profiles:
```
This causes that respective object/collection to be assigned `null`.
YamlDotNet feature request covering this behavior can be found
[here][1].
Fixes#89.
[1]: https://github.com/aaubry/YamlDotNet/issues/443
If scores are missing from a CF in either the guide or in the YAML
config, that CF is still synced but no score will be set in any quality
profiles.
The warning message for this was a bit misleading. It made users think
the CF itself would not be synced. The CF is always synced.
The message has been reworded to make this more clear and it is
downgraded from Warning to Informational.
Reason: Users may use a combination of YAML files, some may not have
both radarr and sonarr config sections in them. We should gracefully
pass over these to allow other configs to be processed.
Should the final list still be empty, the program can gracefully exit
having done no work.
Specify `RECYCLARR_APP_DATA` so that every command inherits and uses
that path for its application data. This behaves the same as if you had
specified the `--app-data` option for every invocation of Recyclarr.
Due to the failures related to symlinks in the repo directory that
happened when doing a full directory merge, the migration logic has been
simplified. It now only copies useful YAML files and cache data. The
repo directory is ignored and will need to be re-cloned when the user
runs `recyclarr` next time.
Symlinks are difficult to deal with. At this point, it was still failing
to migrate the `.config/recyclarr/repo` directory. Even though it still
doesn't work 100%, I'm going to leave it as it is and instead simplify
what gets migrated later.
Smarter migration logic that does a directory merge instead of a
straight move. This is designed to fail less in cases like the
`recyclarr` directory already existing.
Automatic migration no longer takes place. Instead, the user must run
`recyclarr migrate` to have those migration steps executed
automatically, or do it manually.
The logger, which also writes to a file in addition to console,
requires `IAppPaths` in order to find the directory to place the log
files. However, this cannot be obtained until the system calculates
the app data directory OR the user specifies it with the `--app-data`
option.
A custom sink has been added that will allow the logger to write to
console without file logs until that initialization is performed and
the log directory is available.
- Attempt to detect if `HOME` is defined and available. If not, error
out.
- Attempt to create `$HOME/.config` if `$HOME` is available.
- If logic in code attempts to grab the app data dir path before it's
set up, an exception is thrown.
Tests fail inconsistently between Linux & Windows if you hard-code paths
with forward or backward slashes. My hope was that `MockFileSystem`
converted the slashes for me, to assist in simpler unit testing.
The default is now located at `~/.config/recyclarr/recyclarr.yml`. The
previous location (next to the executable) is still supported, but
deprecated. A `recyclarr.yml` at the old/previous location will always
be loaded first.
There were some corner cases that were not handled (such as logging the
instance URL being processed). Additionally, code was simplified greatly
by centralizing the sanitization logic.
The goal is to separate initialization logic from command business
logic. Some initialization requires modifying the environment before we
instantiate many objects needed for implementing command behavior. If
those objects get instantiated, they will most likely already start
using files/directories/environment on the system and we can't modify
those while they're in use.
When using filters like `exclude`, it was possible for terms to not get
synced when they should have. This was due to a misunderstanding of how
`ExceptBy()` and `IntersectBy()` work. According to [an issue][1] on the
dotnet runtime repo, this is by design. The fix is to just avoid those
in favor of `Where()`.
Fixes#69.
[1]: https://github.com/dotnet/dotnet-api-docs/issues/7656
The removal of the markdown parsing logic in v2.0 accidentally also
deleted the logic responsible for handling this property. The code has
been refactored to introduce a "filter pipeline" system that handles
include/exclude filtering as well as strict negative score support.
When ILogger gets instantiated before the Migration steps are processed,
it causes the `recyclarr` directory to be prematurely created, resulting
in an error in the `MigrateTrashUpdaterAppDataDir` migration step
because the destination `recyclarr` directory already exists.
Also:
- New "migration" system which helps perform upgrade steps if needed
- Migration step to rename `trash.yml` to `recyclarr.yml`
- Fixup the `create-config` template
New `--list-terms` command line option which can be used get a list of
terms for a release profile. These lists of terms can be used to include
or exclude specific optionals, for example.
Can be used to quickly and conveniently get a list of release profiles
(and their Trash IDs) so you know what to add in your YAML config under
`release_profiles`.
Previously, Trash Updater would crawl & parse the Trash Guide's markdown
files to obtain information about release profiles. This is complex and
error prone. Thanks to work done by Nitsua, we now have JSON files
available that describe release profiles in a more concise way. These
files are located at `docs/json/sonarr` in the [Trash Guide repo][1].
All of the markdown parsing code has been removed from Trash Updater.
Now, it shares the same git clone of the Trash Guide repository
originally used for Radarr custom formats to access those release
profile JSON files.
BREAKING CHANGE: The old `type:` property for release profiles is
removed in favor of `trash_id:`, which identifies a specific JSON file
to pull data from. Users are required to update their `trash.yml` and
other configuration files to use the new schema. Until changes are made,
users will see errors when they run `trash sonarr` commands.
[1]: https://github.com/TRaSH-/Guides/tree/master/docs/json/sonarr
I observed that when the user specifies an invalid URL to their Radarr
or Sonarr instance (such as forgetting to use the Base URL), it attempts
to redirect by returning HTTP 300. This is now prevented and a warning
is printed to output.
When `IncludeOptionals` was set to false, release profiles with only
optional terms in them would fail to sync to Sonarr because they were
empty. The filter profile logic now considers this config setting and if
set to false, it will ensure that optionals-only release profiles get
filtered out and not pushed to Sonarr.
The Trash sonarr guide was restructured so that optionals were in a
dedicated release profile that had no non-optional terms in it. Logic
was only checking if a profile had non-optional terms in it, and if not,
it got tossed out. Logic now also checks to make sure there are no
optional terms as well.
GitRepository now lazily-constructs its internal Repository object. This
makes the class testable since it no longer does a lot of work in its
constructor. Previously, the string passed to it had to be valid for the
Repository object to accept it.
Code for cloning the repository refactored to handle failures better.
Namely this means deleting the repository and cloning it again if there
is a failure.
Useful in cases where Sonarr or Radarr use HTTPS with a self-signed
certificate. Normally communication with such an instance would fail
since Trash Updater, by default, validates with certificate authorities.
This new setting allows you to disable certificate validation when
communicating with Sonarr or Radarr, avoiding the need to add a
self-signed certificate to your certification store. Use at your own
risk.
Fixes#20.
Registering this type fails the unit tests because it cannot be
constructed unless you use Lazy<> otherwise it throws due to the active
service command not being set yet in the IActiveServiceCommandProvider
class.
The Sonarr developers made a backward-breaking API change resulting in
Trash Updater being unable to obtain, create, or update release
profiles. This fix keeps backward compatibility with the previous and
current schema at the cost of additional code complexity.
The specific breakage was in the Ignored and Required properties of the
Release Profile JSON schema. They were converted from string type to
array.
Offending change:
deed85d2f9/src/NzbDrone.Core/Datastore/Migration/162_release_profile_to_array.csFixes#16
Move non-CLI specific code from Trash project to TrashLib. This is for
future code sharing with Recyclarr. Trash project is officially
deprecated and will eventually go away in favor of the web app.
A new setting under `quality_profiles` of the custom format listing
named `reset_unmatched_scores` that allows the user to specify if
unmapped scores (those CFs not specified in config) should be reset to 0
during quality profile updates.
Fixes#10.
Trash has started putting scores in the actual importable JSON data. If
a score is found in the JSON data, that takes precedence over any score
parsed from the guide markdown. The `trash_score` field is not required
for backward compatibility.
There was an ambiguity with one custom format named 'DoVi'. This one had
two custom formats in the guide. The intent was for the user to choose
only one of these, but the name was kept identical so that name appeared
in the filename for media when it was renamed.
This allows the user to choose which of those two they want using the
`trash_id` property.
Add a version number field to the custom format cache object. This is an
attempt at future-proofing the cache in case I ever need to update the
schema in a non-backward compatible way.
- Synchronize custom formats to Radarr
- Quality profiles can be assigned scores from the guide
- Deletion support for custom formats removed from config or the guide.
- Caching system for keeping track of Custom Format IDs and Trash IDs to
better support renames, deletions, and other stateful behavior.
Also added some code sharing between Radarr and Sonarr for this stuff,
since they are largely similar (Radarr has preferred size but the rest
is the same).
This allows for code to access the active IServiceCommand object, which
represents a subcommand that provides an implementation for a service
like Radarr or Sonarr.
- Executable is now compiled using Ready to Run. This increases the size
of the executable but makes the code much faster.
- `src` directory is no longer the cwd
- The matrix build in build.yml now runs on its respective platform to
avoid cross compilation. Cross compiling does not work with the
ReadyToRun optimization on, see:
https://docs.microsoft.com/en-us/dotnet/core/deploying/ready-to-run#cross-platformarchitecture-restrictions
- publish and zip steps in the workflow have been put in a powershell
script for reusability and to keep the workflow YAML minimal.
Terms marked "optional" (preferred, ignored, and required) are now
separated out from the main list of terms. Right now, any optional terms
are NOT uploaded to Sonarr. In the future, I plan to add ways to
explicitly include optional terms.
The structure for optional terms in the guide is similar to that of
categories. A header OR line within the header section can mention the
word "optional" and that means any code blocks past that point until the
end of the section are treated as optional.
Other changes:
- Delete test trash.yml
- Add new ScopedState class
Used to manage resetting certain parser state between different
sections of the guide (for single code blocks or whole header
sections)
A potential score is one where a number between brackets is found (e.g.
[100]) but the word "score" is not before it. A warning is logged to
notify the user of a potential issue with the guide itself.
Other changes:
- `Serilog.Sinks.TestCorrelator` package added to test log output
messages.
- New StringUtils class in TestLibrary
- Refactored `Regex.Match()` calls to use new extension method that
allows the match to be directly used as an expression in an `if`
condition while also providing the match object as output.
If a tag was not specified in the YAML configuration, an exception would
occur. Logic has been added to properly check for (and allow) empty tags
elements.
The `--strict-negative-scores` option will take any negative preferred
term scores and move those terms to the "Must Not Contains" (ignored)
field of the release profile.
The hybrid quality is auto-generated and takes values between the anime
and non-anime tables in the following way:
* The lowest minimum and highest maximum between the non-anime and anime
tables are used as the min/max for each quality in the generated
hybrid table.
* Only 720/1080 qualities are "combined". Qualities outside of this,
such as 2160, use the non-anime min/max values.
From the Quality Definitions (File Sizes) page of the TRaSH guides, you
can now sync the anime or non-anime quality profile to your Sonarr
instance. There is also a hybrid mode but that is not yet implemented.
Robert Dailey
snoopy82481
Jonas Geiler