Skip to content
Commits on Source (307)
207 additional commits have been omitted to prevent performance issues.
Show whitespace changes
Inline Side-by-side
.gitignore
View file @ c6cfd4eb
......@@ -5,3 +5,4 @@ cover/
*.egg-info/
dist/
build/
test-res/
\ No newline at end of file
.gitlab-ci.yml
View file @ c6cfd4eb
stages:
- mirror
- test
- stats
- release
default:
image: namboy94/ci-docker-environment:0.8.0
before_script:
- echo "$SERVER_ACCESS_KEY" > ~/.ssh/id_rsa
- chmod 0600 ~/.ssh/id_rsa
# TODO Remove this once development stabilizes
- git clone https://gitlab.namibsun.net/namibsun/python/puffotter -b develop
- cd puffotter
- python setup.py install
- cd ..
- rm -rf puffotter
github_mirror:
stage: mirror
tags: [docker]
only: [master, develop]
before_script:
- echo "$GITHUB_SSH_KEY" > ~/.ssh/id_rsa
- chmod 0600 ~/.ssh/id_rsa
script:
- git-mirror-pusher git@github.com:namboy94/toktokkie.git
master develop
stylecheck:
stage: test
tags:
- python3
tags: [docker]
script:
- python3 -m venv virtual && source virtual/bin/activate && pip install ci-scripts
- python-codestyle-check
- python-codestyle-check --exclude toktokkie/gui/pyuic
type_check:
stage: test
tags: [docker]
script:
- python-static-type-check
unittest:
stage: test
tags:
- python3
- progstats
tags: [docker]
script:
- python3 -m venv virtual && source virtual/bin/activate && pip install ci-scripts
- pip install PyQt5
- python-unittest
gitstats:
stage: stats
tags: [docker]
script:
- gitstats-gen
docgen:
stage: stats
tags: [docker]
script:
- pip install PyQt5
- sphinx-docgen
release_upload:
stage: release
only:
- master
tags:
- python3
only: [master]
tags: [docker]
script:
- python3 -m venv virtual && source virtual/bin/activate && pip install ci-scripts
- github-release-upload $(cat version) "$(changelog-reader)"
- gitlab-release-upload $(cat version) "$(changelog-reader)"
pypi_upload:
stage: release
only:
- master
tags:
- python3
only: [master]
tags: [docker]
script:
- python3 -m venv virtual && source virtual/bin/activate && pip install ci-scripts
- pypi-upload
gitstats:
stage: stats
tags:
- python3
- gitstats
- progstats
script:
- python3 -m venv virtual && source virtual/bin/activate && pip install ci-scripts
- gitstats-gen
docgen:
stage: stats
tags:
- python3
- progstats
script:
- python3 -m venv virtual && source virtual/bin/activate && pip install ci-scripts
- sphinx-docgen
CHANGELOG
View file @ c6cfd4eb
V 0.18.0:
- Redid metadata to be more minimalistic and easier to maintain
- Substituted verification for Checkers
- Is now more opinionated, removed options for renaming schemes
- Added CLI tools for adding multi-episodes and excludes
- Implemented toktokkie-supercut tool
- Implemented manga, light novel and visual novel support
- Renamer now supports renaming titles for series with anilist IDs
V 0.17.0:
- Added verification mechanisms
- Added unit tests for metadata and verification
V 0.16.1:
- Removed __init__ file in bin
V 0.16.0:
......
README.md
View file @ c6cfd4eb
# Tok Tokkie Media Manager
# toktokkie
|master|develop|
|:----:|:-----:|
......@@ -6,7 +6,32 @@
![Logo](resources/logo/logo-readme.png)
A collection of command-line media managing tools
The toktokkie media manager consists of a collection of command-line tools used
for keeping track of media.
Currently, the following media types are supported:
- Book
- Book Series
- Movie
- TV Series
- Manga
- Visual Novels
## Structure
The metadata for a Media directory is stored inside the ```.meta```
subdirectory in a file called ```info.json```. Additionally, folder icons may
be stored in ```.meta/icons```. Depending on the metadata type, additional
special folder may exist.
## Generating and modifying metadata
To generate metadata for a media directory, run
```toktokkie metadata-gen <media_type> <directories...>```
Metadata can be modified using a text editor or the ```toktokkie metadata-add```
utility.
## Further Information
......
bin/toktokkie-metadata-gen → bin/toktokkie
View file @ c6cfd4eb
#!/usr/bin/env python
"""LICENSE
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
......@@ -20,40 +18,45 @@ along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
import argparse
from toktokkie import Directory
from toktokkie.metadata import metadata_types
from puffotter.init import cli_start, argparse_add_verbosity
from toktokkie import sentry_dsn
from toktokkie.scripts import toktokkie_commands
def main():
def main(args: argparse.Namespace):
"""
The toktokkie-metadata-gen main method
The main function of this script
:param args: The command line arguments
:return: None
"""
media_types = list(map(lambda x: x.type, metadata_types))
for command_cls in toktokkie_commands:
if command_cls.name() == args.command:
command = command_cls(args)
command.execute()
parser = argparse.ArgumentParser()
parser.add_argument("directory",
help="The directory for which to generate "
"a metadata file for")
parser.add_argument("media_type", choices=set(media_types),
help="The media type of the metadata")
args = parser.parse_args()
directory = args.directory
if directory.endswith("/"):
directory = directory.rsplit("/", 1)[0]
def define_parser() -> argparse.ArgumentParser:
"""
:return: The command line parser for this script
"""
parser = argparse.ArgumentParser()
argparse_add_verbosity(parser)
command_parser = parser.add_subparsers(required=True, dest="command")
media_type = args.media_type
metadata_class = list(filter(
lambda x: x.type == media_type,
metadata_types
))[0]
for command_cls in toktokkie_commands:
subparser = command_parser.add_parser(command_cls.name())
argparse_add_verbosity(subparser)
command_cls.prepare_parser(subparser)
Directory(directory, True, metadata_class)
return parser
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("Thanks for using toktokkie!")
cli_start(
main,
define_parser(),
"Thanks for using toktokkie!",
"toktokkie",
sentry_dsn
)
bin/toktokkie-anime-checker deleted 100644 → 0
View file @ 96cfa385
#!/usr/bin/env python
"""LICENSE
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
import os
import sys
import argparse
import tvdb_api
from typing import List, Dict
from colorama import Fore, Style
from malscraper.types.AiringState import AiringState
from malscraper.types.WatchState import WatchState
from malscraper.UserMalAnime import UserMalAnime
from malscraper.MalAnime import MalAnime
from toktokkie import Directory
from toktokkie.metadata.types.MetaType import Int
from toktokkie.renaming import Plex, Renamer, TVDB
from toktokkie.renaming.helper.resolve import resolve_season, get_episode_files
from toktokkie.metadata import AnimeSeries
from toktokkie.metadata.types.Resolution import Resolution
from toktokkie.metadata.types.AnimeSeriesSeason import AnimeSeriesSeason
from toktokkie.exceptions import InvalidMetadataException, \
MissingMetadataException
def main():
"""
The toktokkie-anime-checker main method
:return: None
"""
parser = argparse.ArgumentParser()
parser.add_argument("directories", nargs="+",
help="The directories to check.")
parser.add_argument("-u", "--mal-username",
help="Your myanimelist.net username")
args = parser.parse_args()
for path in args.directories:
try:
directory = Directory(path)
print(directory.metadata.name)
if args.mal_username is not None:
check_myanimelist(directory, args.mal_username)
check_tvdb_episode_names(directory)
check_tvdb_completeness(directory, args.mal_username)
check_icons(directory)
check_all_directories_in_metadata(directory)
except (InvalidMetadataException, MissingMetadataException) as e:
print("Invalid Metadata")
print(str(e))
sys.exit(1)
print("Done!")
def check_icons(directory: Directory):
"""
Checks that icons exist for all applicable directories
:param directory: The directory to check
:return: None
"""
for child in os.listdir(directory.path):
child_path = os.path.join(directory.path, child)
if os.path.isdir(child_path) and not child.startswith("."):
icon_path = os.path.join(directory.icon_path, child + ".png")
if not os.path.isfile(icon_path):
print(Fore.LIGHTWHITE_EX + "No Icon for " + child_path +
Style.RESET_ALL)
if not os.path.isfile(os.path.join(directory.icon_path, "main.png")):
print(Fore.LIGHTWHITE_EX + "No main icon for " +
directory.metadata.name + Style.RESET_ALL)
def check_all_directories_in_metadata(directory: Directory):
"""
Makes sure that all subdirectories of a media folder has a
season entry in the metadata file
:param directory: The directory to check
:return: None
"""
meta = directory.metadata
for child in os.listdir(directory.path):
child_path = os.path.join(directory.path, child)
if os.path.isdir(child_path) and not child.startswith("."):
search = list(filter(lambda x: x.path == child, meta.seasons.list))
if len(search) != 1:
print("No metadata for " + child_path)
def check_tvdb_episode_names(directory: Directory):
"""
Checks that the episode names are all correct according to TheTVDB.com
:param directory: The media directory to check
:return: None
"""
metadata = AnimeSeries.from_json_file(directory.metadata_file)
renamer = Renamer(directory.path, metadata, Plex, TVDB)
episodes = renamer.episodes
valid = True
longest = len(max(episodes, key=lambda x: len(x.current)).current)
for episode in episodes:
if episode.current != episode.new:
if valid:
print("Invalid Episodes [" + metadata.name + "]:")
print(Fore.LIGHTMAGENTA_EX +
episode.current.ljust(longest + 1) + Style.RESET_ALL +
" != " + Fore.LIGHTBLUE_EX + episode.new +
Style.RESET_ALL)
valid = False
if not valid:
print("Suggestion:")
renamer.rename(False)
def check_tvdb_completeness(directory: Directory, mal_username: str):
"""
Checks if all tvdb episodes are included or at least excluded/in multis
:param directory: The directory to check
:param mal_username: The myanimelist usernam for MAL crosscheck
:return: None
"""
tvdb = tvdb_api.Tvdb()
episode_count = {}
file_counts = {}
# Only one tvdb id for now
tvdb_id = directory.metadata.seasons.list[0].tvdb_ids.list[0]
tvdb_data = tvdb[tvdb_id]
for season_number, season_data in tvdb_data.items():
file_counts[season_number] = 0
episode_count[season_number] = 0
for _ in season_data:
episode_count[season_number] += 1
for season in directory.metadata.seasons.list:
season_path = os.path.join(directory.path, season.path)
season_number = resolve_season(season_path)
file_counts[season_number] += len(get_episode_files(season_path))
if mal_username is not None:
# TODO
pass
for ex in directory.metadata.tvdb_excludes.list:
file_counts[ex.season] += 1
for multi in directory.metadata.tvdb_multi_episodes.list:
file_counts[multi.start.season] += multi.diff()
for season, count in episode_count.items():
if file_counts[season] != count:
print(Fore.WHITE + "TVDB Season " + str(season) +
" missing local files. (Have: " + str(file_counts[season]) +
", Need: " + str(count) + ")" + Style.RESET_ALL)
def check_myanimelist(directory: Directory, username: str):
"""
Checks if the data is correct as documented on myanimelist
:param directory: The directhttp://192.168.1.2/ory to check
:param username: Provide a myanimelist user for more checks
:return: None
"""
metadata = directory.metadata
mal_map = {}
for season in metadata.seasons.list:
for mal_id in season.mal_ids.list:
if mal_id in mal_map:
mal_map[mal_id].append(season)
else:
mal_map[mal_id] = [season]
# Include related IDs
related_ids = []
for mal_id in mal_map:
mal_data = MalAnime(mal_id)
for _id in mal_data.related_anime:
if _id not in related_ids:
related_ids.append(_id)
for ignored in metadata.mal_check_ignores.list:
if ignored in related_ids:
related_ids.remove(ignored)
for related in related_ids:
if related not in mal_map:
mal_map[related] = []
if not check_myanimelist_watch_state(directory, mal_map, username):
return
if not check_myanimelist_watch_dates(mal_map, username):
return
if not check_myanimelist_tags(mal_map, username):
return
if not check_myanimelist_local_files(directory, mal_map, username):
return
def check_myanimelist_local_files(directory: Directory,
mal_map: Dict[int, List[AnimeSeriesSeason]],
username: str) -> bool:
"""
Checks if the local files are matching with the myanimelist data
:param directory: The directory containing the files
:param mal_map: The myanimelist IDs mapped to lists of seasons
:param username: The myanimelist username
:return: True if everything is fine, False otherwise
"""
valid = True
for mal_id, seasons in mal_map.items():
mal_data = UserMalAnime(mal_id, username)
if mal_data.watch_status != WatchState.COMPLETED:
continue
if len(seasons) == 0:
print(Fore.LIGHTMAGENTA_EX + "Myanimelist ID " + str(mal_id) + " ("
+ mal_data.name + ") missing from local files" +
Style.RESET_ALL)
valid = False
continue
episode_count = calculate_episode_amount(mal_id, directory, mal_map)
if mal_data.episode_count != mal_data.episodes_watched_count:
print(
Fore.LIGHTCYAN_EX +
"Episodes available and watched out of sync for " +
mal_data.name + " (" + str(mal_id) + "): [" +
str(mal_data.episode_count) + "|" +
str(mal_data.episodes_watched_count) + "]" + Style.RESET_ALL
)
valid = False
if mal_data.episode_count != episode_count:
print(
Fore.LIGHTBLUE_EX +
"Episodes available and local files out of sync for " +
mal_data.name + " (" + str(mal_id) + "): [" +
str(mal_data.episode_count) + "|" + str(episode_count) +
"]" + Style.RESET_ALL
)
valid = False
return valid
def check_myanimelist_watch_state(directory: Directory,
mal_map: Dict[int, List[AnimeSeriesSeason]],
username: str) -> bool:
"""
Makes sure that all myanimelist IDs provided have been marked as completed,
provided they have already finished airing
:param directory: The directory to use
:param mal_map: The myanimelist IDs mapped to lists of seasons
:param username: The myanimelist username
:return: True if everything is fine, False otherwise
"""
valid = True
for mal_id in mal_map:
mal_data = UserMalAnime(mal_id, username)
to_print = "Not Completed: " + mal_data.name + " [" + str(mal_id) + "]"
if mal_data.airing_status == AiringState.FINISHED:
if mal_data.watch_status != WatchState.COMPLETED:
if mal_data.watch_status == WatchState.PLAN_TO_WATCH:
continue
elif mal_data.watch_status == WatchState.ON_HOLD:
print(Fore.GREEN + to_print + Style.RESET_ALL)
else:
print(Fore.LIGHTRED_EX + to_print + Style.RESET_ALL)
if input("Ignore this entry? (y/n)") == "y":
directory.metadata.mal_check_ignores.list.append(
Int(mal_id)
)
directory.write_metadata()
print("Ignoring entry " + str(mal_id))
else:
print("Not ignoring entry " + str(mal_id))
valid = False
return valid
def check_myanimelist_watch_dates(mal_map: Dict[int, List[AnimeSeriesSeason]],
username: str) -> bool:
"""
Checks if the user-entered watching dates are entered for every completed
item
:param mal_map: The myanimelist IDs mapped to lists of seasons
:param username: The username to use
:return: True if everything is fine, False otherwise
"""
valid = True
for mal_id in mal_map:
mal_data = UserMalAnime(mal_id, username)
if mal_data.watch_status == WatchState.COMPLETED:
if mal_data.start_watching_date is None \
or mal_data.finish_watching_date is None:
print(Fore.MAGENTA + "Watch dates missing for " + mal_data.name
+ "[" + str(mal_id) + "]" + Style.RESET_ALL)
valid = False
return valid
def check_myanimelist_tags(mal_map: Dict[int, List[AnimeSeriesSeason]],
username: str) -> bool:
"""
Checks that the myanimelist user-added tags actually confirm the data in the
metadata
:param mal_map: The myanimelist IDs mapped to lists of seasons
:param username: The username
:return: True if everything is fine, False otherwise
"""
valid = True
for mal_id, seasons in mal_map.items():
mal_data = UserMalAnime(mal_id, username)
if mal_data.watch_status != WatchState.COMPLETED:
continue
if len(mal_data.tags) < 2:
print(Fore.YELLOW + "Not enough tags for " + mal_data.name +
"[" + str(mal_id) + "]" + Style.RESET_ALL)
valid = False
for tag in mal_data.tags:
if tag.lower() in ["subbed", "dubbed"]:
audios = []
subtitles = []
if tag.lower() == "subbed":
audios.append("jpn")
subtitles.append("eng")
if tag.lower() == "dubbed":
audios.append("eng")
for season in seasons:
for audio in audios:
if audio not in season.audio_langs.to_json():
print(Fore.YELLOW + "Missing audio: " + str(audio) +
" for " + mal_data.name + Style.RESET_ALL)
valid = False
for subtitle in subtitles:
if subtitle not in season.subtitle_langs.to_json():
print(Fore.YELLOW + "Missing audio: " +
str(subtitle) + " for " + mal_data.name +
Style.RESET_ALL)
valid = False
else:
try:
resolution = Resolution.parse(tag)
for season in seasons:
if resolution.to_json() \
not in season.resolutions.to_json():
print(Fore.YELLOW + "Resolution " + str(resolution)
+ " missing for " + mal_data.name +
" in local metadata." + Style.RESET_ALL)
valid = False
except (IndexError, ValueError):
pass
return valid
def calculate_episode_amount(mal_id: int, directory: Directory,
mal_map: Dict[int, List[AnimeSeriesSeason]]) \
-> int:
"""
Calculates the amount of episodes that are present locally for a given
myanimelist ID.
This is definitely the most complex function in this entire script
:param mal_id: The myanimelist ID
:param directory: The directory in which to search
:param mal_map: Dictionary mapping myanimelist ids to seasons
:return: The episode amount
"""
season_episodes = {mal_id: []}
unique_id = "/////"
# Collect all RELATED myanimelist IDs
for season in mal_map[mal_id]:
for season_mal_id in season.mal_ids.list:
if season_mal_id != mal_id:
season_episodes[season_mal_id] = []
# Search for all episodes related to a specific myanimelist id
for season in directory.metadata.seasons.list:
season_dir = os.path.join(directory.path, season.path)
for season_mal_id in season_episodes:
if season_mal_id in season.mal_ids.to_json():
for episode in os.listdir(season_dir):
episode_path = os.path.join(season_dir, episode)
if os.path.isfile(episode_path) and \
not episode.startswith("."):
season_episodes[season_mal_id].append(episode_path)
# Add excluded episodes
for excluded in directory.metadata.mal_excludes.list:
if excluded.mal_id == season_mal_id:
season_episodes[season_mal_id].append(unique_id)
unique_id += "/"
# Add multi episodes
for multi_episode in directory.metadata.mal_multi_episodes.list:
if multi_episode.start.mal_id == season_mal_id:
diff = abs(multi_episode.end.episode -
multi_episode.start.episode)
for _ in range(0, diff):
season_episodes[season_mal_id].append(unique_id)
unique_id += "/"
# Figure out how many episodes are there in total,
# across all related myanimelist episodes
all_episodes = []
for _, episodes in season_episodes.items():
all_episodes += episodes
episode_count = len(set(all_episodes))
# Subtract all episodes from other IDs
for season_mal_id in season_episodes:
if season_mal_id != mal_id:
season_mal_data = MalAnime(season_mal_id)
episode_count -= season_mal_data.episode_count
return episode_count
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("Thanks for using toktokkie!")
bin/toktokkie-rename deleted 100644 → 0
View file @ 96cfa385
#!/usr/bin/env python
"""LICENSE
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
import argparse
from toktokkie.renaming import schemes, agents, Plex, TVDB
from toktokkie import Directory
def main():
"""
The toktokkie-rename main method
:return: None
"""
naming_schemes = list(map(lambda x: x.name, schemes))
naming_agents = list(map(lambda x: x.name, agents))
default_scheme = Plex.name
default_agent = TVDB.name
parser = argparse.ArgumentParser()
parser.add_argument("directories", nargs="+",
help="The directories to rename. "
"Files and directories that do not contain any "
"valid metadata configuration will be ignored.")
parser.add_argument("scheme", choices=set(naming_schemes), nargs="?",
default=default_scheme,
help="The naming scheme to use")
parser.add_argument("agent", choices=set(naming_agents), nargs="?",
default=default_agent,
help="The episode data fetching agent to use")
parser.add_argument("--noconfirm", action="store",
help="Skips the user confirmation step")
args = parser.parse_args()
directories = args.directories
scheme = list(filter(lambda x: x.name == args.scheme, schemes))[0]
agent = list(filter(lambda x: x.name == args.agent, agents))[0]
for path in directories:
try:
directory = Directory(path)
directory.rename(scheme, agent, args.noconfirm)
except ValueError:
print("Renaming of " + path + "failed")
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("Thanks for using toktokkie!")
bin/toktokkie-xdcc-update deleted 100644 → 0
View file @ 96cfa385
#!/usr/bin/env python
"""LICENSE
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
import argparse
from xdcc_dl.helper import set_throttle_value
from toktokkie.renaming import schemes, agents, Plex, TVDB
from toktokkie import Directory
from toktokkie.exceptions import MissingMetadataException, \
MissingUpdateInstructionsException
def main():
"""
The toktokkie-xdcc-update main method
:return: None
"""
naming_schemes = list(map(lambda x: x.name, schemes))
naming_agents = list(map(lambda x: x.name, agents))
default_scheme = Plex.name
default_agent = TVDB.name
parser = argparse.ArgumentParser()
parser.add_argument("directories", nargs="+",
help="The directories to xdcc-update. "
"Files and directories that do not contain any "
"valid metadata configuration will be ignored.")
parser.add_argument("scheme", choices=set(naming_schemes), nargs="?",
default=default_scheme,
help="The naming scheme to use")
parser.add_argument("agent", choices=set(naming_agents), nargs="?",
default=default_agent,
help="The episode data fetching agent to use")
parser.add_argument("-c", "--create", action="store_true",
help="If set, will prompt for information and create "
"new xdcc update instructions")
parser.add_argument("-t", "--throttle",
help="Sets a speed at which to throttle the download")
args = parser.parse_args()
directories = args.directories
scheme = list(filter(lambda x: x.name == args.scheme, schemes))[0]
agent = list(filter(lambda x: x.name == args.agent, agents))[0]
set_throttle_value(args.throttle)
for path in directories:
try:
directory = Directory(path)
print(directory.metadata.name)
if args.create:
directory.xdcc_update(scheme, agents, True)
else:
directory.xdcc_update(scheme, agent)
except ValueError as e:
print(e)
print("Updating of " + path + "failed")
raise e
except MissingMetadataException:
print("Missing metadata: " + path)
except MissingUpdateInstructionsException:
print("Missing update instructions. Run with --create first")
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("Thanks for using toktokkie!")
doc/brainstorm/check.txt 0 → 100644
View file @ c6cfd4eb
list | data
list only:
- check relations
- if not completed and not done:
if "cantfind" in tags:
skip
else:
error
- check start/end dates
- check score
both:
- for entry in list: check if ID in data
- for item in data:
for season in item:
check episode counts.
\ No newline at end of file
qt_refresh.py 0 → 100755
View file @ c6cfd4eb
#!/bin/env python
"""LICENSE
Copyright 2019 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie-gui. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
import os
from subprocess import check_output
qt_designer_dir = os.path.join("toktokkie", "gui", "qt_designer")
pyuic_dir = os.path.join("toktokkie", "gui", "pyuic")
for design in os.listdir(qt_designer_dir):
if design.endswith(".ui"):
design_file = os.path.join(qt_designer_dir, design)
result_file = os.path.join(pyuic_dir, design.replace(".ui", ".py"))
generated = check_output(["pyuic5", design_file])
with open(result_file, "wb") as f:
f.write(generated)
setup.py
View file @ c6cfd4eb
......@@ -44,8 +44,16 @@ if __name__ == "__main__":
"requests",
"xdcc_dl",
"colorama",
"malscraper"
"anime_list_apis",
"mutagen",
"puffotter",
"manga-dl"
],
extras_require={
"gui": ["PyQt5"]
},
test_suite='nose.collector',
tests_require=['nose'],
include_package_data=True,
zip_safe=False
)
toktokkie/Directory.py
View file @ c6cfd4eb
......@@ -19,12 +19,14 @@ LICENSE"""
import os
import sys
from typing import Type
from toktokkie.renaming import Renamer, Scheme, Agent
from toktokkie.iconizing import Iconizer, Procedure
from toktokkie.metadata import resolve_metadata, Base, TvSeries
from toktokkie.exceptions import MissingMetadataException
from typing import Dict, Any
from toktokkie.renaming.Renamer import Renamer
from toktokkie.iconizing.Iconizer import Iconizer, Procedure
from toktokkie.metadata.helper.functions import get_metadata, create_metadata
from toktokkie.metadata.components.enums import MediaType
from toktokkie.exceptions import MissingMetadata
from toktokkie.xdcc_update.XDCCUpdater import XDCCUpdater
from toktokkie.check.map import checker_map
class Directory:
......@@ -33,14 +35,16 @@ class Directory:
"""
def __init__(self, path: str, generate_metadata: bool = False,
metadata_type: any = None):
metadata_type: str = None):
"""
Initializes the metadata of the directory
:param path: The directory's path
:except MissingMetadataException,
InvalidMetadataException,
MetadataMismatch
"""
self.path = path
self.meta_dir = os.path.join(path, ".meta")
self.icon_path = os.path.join(self.meta_dir, "icons")
self.metadata_file = os.path.join(self.meta_dir, "info.json")
if generate_metadata:
......@@ -50,20 +54,28 @@ class Directory:
self.generate_metadata(metadata_type)
if not os.path.isfile(self.metadata_file):
raise MissingMetadataException(self.metadata_file + " missing")
self.metadata = resolve_metadata(self.metadata_file)
raise MissingMetadata(self.metadata_file + " missing")
if not os.path.isdir(self.icon_path):
os.makedirs(self.icon_path)
self.metadata = get_metadata(self.path)
if not os.path.isdir(self.metadata.icon_directory):
os.makedirs(self.metadata.icon_directory)
def reload(self):
"""
Reloads the metadata from the metadata file
:return: None
"""
self.metadata = get_metadata(self.path)
def write_metadata(self):
"""
Updates the metadata file with the current contents of the metadata
:return: None
"""
self.metadata.write(self.metadata_file)
self.metadata.write()
def generate_metadata(self, metadata_type: Base):
def generate_metadata(self, metadata_type: str):
"""
Prompts the user for metadata information
:param metadata_type: The metadata type to generate
......@@ -78,26 +90,17 @@ class Directory:
print("Aborting")
sys.exit(0)
metadata = metadata_type.generate_from_prompts(self.path) # type: Base
metadata = create_metadata(self.path, metadata_type)
metadata.write()
if not os.path.isdir(self.meta_dir):
os.makedirs(self.meta_dir)
metadata.write(self.metadata_file)
def rename(self, scheme: Type[Scheme], agent: Type[Agent],
noconfirm: bool = False):
def rename(self, noconfirm: bool = False):
"""
Renames the contained files according to a naming scheme.
If the metadata type does not support renaming, this does nothing
:param scheme: The naming scheme to use
:param agent: The data gathering agent to use
Renames the contained files.
:param noconfirm: Skips the confirmation phase
:return: None
"""
if self.metadata.is_subclass_of(TvSeries):
# noinspection PyTypeChecker
renamer = Renamer(self.path, self.metadata, scheme, agent)
renamer.rename(noconfirm)
Renamer(self.metadata).rename(noconfirm)
self.path = self.metadata.directory_path
def iconize(self, procedure: Procedure):
"""
......@@ -105,18 +108,39 @@ class Directory:
:param procedure: The iconizing procedure to use
:return: None
"""
iconizer = Iconizer(self.path, self.icon_path, procedure)
iconizer = Iconizer(self.path, self.metadata.icon_directory, procedure)
iconizer.iconize()
def xdcc_update(self, scheme: Type[Scheme], agent: Type[Agent],
create: bool = False):
def check(
self,
show_warnings: bool,
fix_interactively: bool,
config: Dict[str, Any]
) -> bool:
"""
Performs a check, making sure that everything in the directory
is configured correctly and up-to-date
:param show_warnings: Whether or not to show warnings
:param fix_interactively: Whether or not to enable interactive fixing
:param config: Configuration dictionary for checks
:return: The check result
"""
checker_cls = checker_map[self.metadata.media_type()]
checker = checker_cls(
self.metadata,
show_warnings,
fix_interactively,
config
)
return checker.check()
def xdcc_update(self):
"""
Performs an XDCC Update Action
:param scheme: The naming scheme to use
:param agent: The naming agent to use
:param create: Can be set to create the XDCC Update instructions
:return: None
"""
updater = XDCCUpdater(self.path, self.metadata, scheme, agent, create)
if not create:
updater.update()
if self.metadata.media_type() == MediaType.TV_SERIES:
# noinspection PyTypeChecker
XDCCUpdater(self.metadata).update()
else:
print("xdcc-update is only supported for TV series")
toktokkie/__init__.py
View file @ c6cfd4eb
......@@ -17,4 +17,8 @@ You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
from toktokkie.Directory import Directory
sentry_dsn = "https://77992bd58d9a46fc812ad491ba460a7e@sentry.namibsun.net/10"
"""
Sentry DSN used for exception logging
"""
toktokkie/exceptions/InvalidMetadataException.py → toktokkie/check/BookChecker.py
View file @ c6cfd4eb
"""LICENSE
"""
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
......@@ -15,11 +15,13 @@ GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
"""
from toktokkie.check.Checker import Checker
class InvalidMetadataException(Exception):
class BookChecker(Checker):
"""
Exception that is raised whenever metadata is invalid
Class that check Book media for consistency
"""
pass
toktokkie/check/BookSeriesChecker.py 0 → 100644
View file @ c6cfd4eb
"""
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
"""
from toktokkie.check.Checker import Checker
from toktokkie.metadata.BookSeries import BookSeries
from toktokkie.metadata.components.enums import IdType
from anime_list_apis.models.attributes.Id import IdType as AnimeListIdType
class BookSeriesChecker(Checker):
"""
Class that check Book Series media for consistency
"""
def check(self) -> bool:
"""
Performs sanity checks and prints out anything that's wrong
:return: The result of the check
"""
valid = super().check()
if self.config.get("anilist_user") is not None:
valid = self._check_anilist_read_state() and valid
return valid
def _check_anilist_read_state(self) -> bool:
"""
Checks if the anilist user is up-to-date with all available volumes
:return: The check result
"""
metadata = self.metadata # type: BookSeries # type: ignore
manga_list = self.config["anilist_manga_list"]
try:
_id = metadata.ids.get(IdType.ANILIST, [])[0]
except IndexError:
return self.error("No Anilist ID")
anilist_id = int(_id)
manga = None
for entry in manga_list:
if entry.id.get(AnimeListIdType.ANILIST) == anilist_id:
manga = entry
break
if manga is None:
return self.error("Not in Anilist")
else:
volumes_local = len(metadata.volumes)
volumes_read = manga.volume_progress
if volumes_read < volumes_local:
return self.warn("User has only read {}/{} volumes".format(
volumes_read, volumes_local
))
elif volumes_local < volumes_read:
return self.warn(
"Some read volumes do not exist locally "
"(Read:{}, Local:{})".format(
volumes_read, volumes_local
))
else:
return True
toktokkie/check/Checker.py 0 → 100644
View file @ c6cfd4eb
"""
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
"""
import os
import tvdb_api
from typing import Dict, Any
from colorama import Fore, Back, Style
from toktokkie.metadata.Metadata import Metadata
from toktokkie.renaming.Renamer import Renamer
from anime_list_apis.api.AnilistApi import AnilistApi
class Checker:
"""
Class that performs checks on Metadata objects
"""
def __init__(
self,
metadata: Metadata,
show_warnings: bool,
fix_interactively: bool,
config: Dict[str, Any]
):
"""
Initializes the checker
:param metadata: The metadata to check
:param show_warnings: Whether or not to show warnings
:param fix_interactively: Can be set to True to
interactively fix some errors
:param config: A dictionary containing configuration options
"""
self.metadata = metadata
self.show_warnings = show_warnings
self.fix_interactively = fix_interactively
self.config = config
self.tvdb = config.get("tvdb_api", tvdb_api.Tvdb())
self.anilist_user = config.get("anilist_user")
self.anilist_api = config.get("anilist_api", AnilistApi())
self.anilist_anime_list = config.get("anilist_anime_list", [])
self.anilist_manga_list = config.get("anilist_manga_list", [])
def check(self) -> bool:
"""
Performs sanity checks and prints out anything that's wrong
:return: The result of the check
"""
print("-" * 80)
print("Checking {}".format(self.metadata.name))
valid = self._check_icons()
valid = self._check_renaming() and valid
return valid
def _check_icons(self) -> bool:
"""
Checks that the icon directory exists and there's an icon file for
every child directory as well as the main directory.
:return: The result of the check
"""
valid = True
if not os.path.isdir(self.metadata.icon_directory):
valid = self.error("Missing icon directory")
main_icon = os.path.join(self.metadata.icon_directory, "main.png")
if not os.path.isfile(main_icon):
valid = self.error("Missing main icon file for {}".format(
self.metadata.name
))
for child in os.listdir(self.metadata.directory_path):
child_path = os.path.join(self.metadata.directory_path, child)
if child.startswith(".") or not os.path.isdir(child_path):
continue
else:
icon_file = os.path.join(
self.metadata.icon_directory, child + ".png"
)
if not os.path.isfile(icon_file):
valid = \
self.error("Missing icon file for {}".format(child))
return valid
def _check_renaming(self) -> bool:
"""
Checks if the renaming of the files and directories of the
metadata content is correct and up-to-date
:return: The result of the check
"""
valid = True
renamer = Renamer(self.metadata)
has_errors = False
for operation in renamer.operations:
if operation.source != operation.dest:
valid = self.error("File Mismatch:")
print("{}{}{}".format(
Fore.LIGHTGREEN_EX,
os.path.basename(operation.source),
Style.RESET_ALL
))
print("{}{}{}".format(
Fore.LIGHTCYAN_EX,
os.path.basename(operation.dest),
Style.RESET_ALL
))
has_errors = True
if has_errors and self.fix_interactively:
renamer.rename(False)
return True
else:
return valid
# noinspection PyMethodMayBeStatic
def error(self, text: str) -> bool:
"""
Prints a black-on-red error message
:param text: The text to print
:return: False
"""
print("{}{}{}{}".format(Back.RED, Fore.BLACK, text, Style.RESET_ALL))
return False
def warn(self, text: str) -> bool:
"""
Prints a black-on-yellow warning message
:param text: The text to print
:return: True if not showing warnings, else False
"""
if self.show_warnings:
print("{}{}{}{}".format(
Back.YELLOW, Fore.BLACK, text, Style.RESET_ALL
))
return not self.show_warnings
toktokkie/check/MangaChecker.py 0 → 100644
View file @ c6cfd4eb
"""
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
"""
import os
import json
import requests
from typing import Optional, List
from toktokkie.check.Checker import Checker
from toktokkie.metadata.Manga import Manga
from toktokkie.metadata.components.enums import IdType
from anime_list_apis.api.AnilistApi import AnilistApi
from anime_list_apis.models.MediaListEntry import MangaListEntry
from anime_list_apis.models.attributes.Id import IdType as AnimeListIdType
class MangaChecker(Checker):
"""
Class that check Manga media for consistency
"""
def check(self) -> bool:
"""
Performs sanity checks and prints out anything that's wrong
:return: The result of the check
"""
valid = super().check()
valid = self._check_special_chapter_count() and valid
if self.config.get("anilist_user") is not None:
valid = self._check_chapter_progress() and valid
return valid
def _check_icons(self) -> bool:
"""
Only checks for a main.png icon file.
:return: The result of the check
"""
valid = True
if not os.path.isdir(self.metadata.icon_directory):
valid = self.error("Missing icon directory")
main_icon = os.path.join(self.metadata.icon_directory, "main.png")
if not os.path.isfile(main_icon):
valid = self.error("Missing main icon file for {}".format(
self.metadata.name
))
return valid
def _check_special_chapter_count(self) -> bool:
"""
Checks if the correct amount of special chapters exist in the special
directory
:return: True if correct, False otherwise
"""
# noinspection PyTypeChecker
metadata = self.metadata # type: Manga # type: ignore
should = len(os.listdir(metadata.special_path))
_is = len(metadata.special_chapters)
if should != _is:
self.error(
"Incorrect amount of special chapters: Should: {}, Is: {}"
.format(should, _is)
)
return False
return True
def _check_chapter_progress(self) -> bool:
"""
Checks the chapter progress using the best guess anilist user data
can give us.
:return: The result of the check
"""
# noinspection PyTypeChecker
metadata = self.metadata # type: Manga # type: ignore
anilist_entries = self.config["anilist_manga_list"]
local_chaptercount = len(os.listdir(metadata.main_path))
try:
anilist_ids = metadata.ids.get(IdType.ANILIST)
if anilist_ids is None:
raise IndexError
anilist_id = int(anilist_ids[0])
list_entry = None
for entry in anilist_entries: # type: MangaListEntry
if entry.id.get(AnimeListIdType.ANILIST) == anilist_id:
list_entry = entry
break
list_chaptercount = None
if list_entry is not None:
list_chaptercount = list_entry.chapter_progress
remote_chaptercount = self._guess_latest_chapter(int(anilist_id))
list_complete = list_chaptercount == local_chaptercount
remote_complete = remote_chaptercount == local_chaptercount
if not list_complete:
self.warn("Local chapters and list chapters "
"don't match: Local: {} / List: {}"
.format(local_chaptercount, list_chaptercount))
if not remote_complete:
self.warn("Local chapters and available chapters "
"don't match: Local: {} / Available: {}"
.format(local_chaptercount, remote_chaptercount))
return list_complete and remote_complete
except (IndexError, ValueError):
self.warn("No Anilist ID for {}".format(metadata.name))
return True
def _guess_latest_chapter(self, anilist_id: int) -> Optional[int]:
"""
Guesses the latest chapter number based on anilist user activity
:param anilist_id: The anilist ID to check
:return: The latest chapter number
"""
api = self.config["anilist_api"] # type: AnilistApi
info = api.get_manga_data(anilist_id)
chapter_count = info.chapter_count
if chapter_count is None: # Guess if not official data is present
query = """
query ($id: Int) {
Page(page: 1) {
activities(mediaId: $id, sort: ID_DESC) {
... on ListActivity {
progress
userId
}
}
}
}
"""
resp = requests.post(
"https://graphql.anilist.co",
json={"query": query, "variables": {"id": anilist_id}}
)
data = json.loads(resp.text)["data"]["Page"]["activities"]
progresses = []
for entry in data:
progress = entry["progress"]
if progress is not None:
progress = entry["progress"].split(" - ")[-1]
progresses.append(int(progress))
progresses.sort()
chapter_count = progresses[-1]
return chapter_count
toktokkie/exceptions/MissingMetadataException.py → toktokkie/check/MovieChecker.py
View file @ c6cfd4eb
"""LICENSE
"""
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
......@@ -15,11 +15,13 @@ GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
LICENSE"""
"""
from toktokkie.check.Checker import Checker
class MissingMetadataException(Exception):
class MovieChecker(Checker):
"""
Exception that is raised whenever metadata is not found
Class that check Movie media for consistency
"""
pass
toktokkie/check/TvSeriesChecker.py 0 → 100644
View file @ c6cfd4eb
"""
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
"""
import os
from tvdb_api import tvdb_shownotfound
from typing import Dict, Optional, List
from datetime import datetime
from colorama import Fore, Style
from toktokkie.check.Checker import Checker
from toktokkie.renaming.Renamer import Renamer
from toktokkie.renaming.RenameOperation import RenameOperation
from toktokkie.metadata.TvSeries import TvSeries
from toktokkie.metadata.components.enums import IdType
from anime_list_apis.api.AnilistApi import AnilistApi
from anime_list_apis.models.attributes.Id import IdType as AnimeListIdType
from anime_list_apis.models.attributes.MediaType import MediaType
from anime_list_apis.models.attributes.ConsumingStatus import ConsumingStatus
from anime_list_apis.models.attributes.ReleasingStatus import ReleasingStatus
class TvSeriesChecker(Checker):
"""
Class that check TV Series media for consistency
"""
def check(self) -> bool:
"""
Performs sanity checks and prints out anything that's wrong
:return: The result of the check
"""
valid = super().check()
valid = self._check_season_metadata() and valid
valid = self._check_tvdb_ids() and valid
# Subsequent steps need to have fully valid metadata and tvdb ids
if not valid:
return valid
valid = self._check_tvdb_season_lengths() and valid
valid = self._check_tvdb_episode_files_complete() and valid
valid = self._check_spinoff_completeness() and valid
if self.config.get("anilist_user") is not None:
valid = self._check_anilist_ids() and valid
return valid
def _check_season_metadata(self) -> bool:
"""
Makes sure that every season directory has a corresponding
metadata entry.
:return: The result of the check
"""
valid = True
metadata = self.metadata # type: TvSeries # type: ignore
for season_name in os.listdir(metadata.directory_path):
season_path = os.path.join(metadata.directory_path, season_name)
if os.path.isfile(season_path) or season_name.startswith("."):
continue
try:
metadata.get_season(season_name)
except KeyError:
valid = self.error(
"Missing metadata for Season '{}'".format(season_name)
)
return valid
def _check_tvdb_ids(self) -> bool:
"""
Makes sure that all TVDB Ids are valid and point to actual entries
on the website.
:return: The result of the check
"""
valid = True
metadata = self.metadata # type: TvSeries # type: ignore
ids = [metadata.tvdb_id]
for season in metadata.seasons:
ids.append(season.tvdb_id)
for tvdb_id in ids:
if int(tvdb_id) == 0: # TVDB ID 0 means show not on tvdb
continue
try:
_ = self.tvdb[int(tvdb_id)]
except tvdb_shownotfound:
valid = self.error("Entry {} not found on TVDB")
return valid
def _check_tvdb_season_lengths(self) -> bool:
"""
Checks that the length of the seasons are in accordance with the
amount of episodes on TVDB.
:return: The result of the check
"""
valid = True
metadata = self.metadata # type: TvSeries # type: ignore
ignores = self._generate_ignores_map()
tvdb_data = self.tvdb[int(metadata.tvdb_id)]
for season_number, season_data in tvdb_data.items():
episode_amount = len(season_data)
for episode_number, episode_data in season_data.items():
# Don't count unaired episodes
if not self._has_aired(episode_data):
episode_amount -= 1
# Subtract ignored episodes
elif episode_number in ignores.get(season_number, []):
episode_amount -= 1
_existing = metadata.get_episode_files()
existing = _existing[metadata.tvdb_id].get(season_number, [])
if not len(existing) == episode_amount:
msg = "Mismatch in season {}; Should:{}; Is:{}".format(
season_number, episode_amount, len(existing)
)
valid = self.error(msg)
return valid
def _check_tvdb_episode_files_complete(self) -> bool:
"""
Makes sure that all episodes entered on thetvdb.com are present
in the directory or otherwise excluded using metadata
:return: The result of the check
"""
valid = True
metadata = self.metadata # type: TvSeries # type: ignore
ignores = self._generate_ignores_map()
tvdb_data = self.tvdb[int(metadata.tvdb_id)]
for season_number, season_data in tvdb_data.items():
for episode_number, episode_data in season_data.items():
if not self._has_aired(episode_data):
continue
# Ignore ignored episode number
if episode_number in ignores.get(season_number, []):
continue
episode_name = self._generate_episode_name(
metadata.tvdb_id, season_number, episode_number
)
# Check if file exists
existing_files = \
metadata.get_episode_files()[metadata.tvdb_id].get(
season_number, []
)
exists = False
for episode_file in existing_files:
existing_name = os.path.basename(episode_file)
if existing_name.startswith(episode_name):
exists = True
break
if not exists:
valid = self.error(
"Episode {} does not exist "
"or is incorrectly named".format(episode_name)
)
return valid
def _check_spinoff_completeness(self) -> bool:
"""
Makes sure that any spinoff series are also available and complete
:return: The result of the check
"""
valid = True
metadata = self.metadata # type: TvSeries # type: ignore
episode_files = metadata.get_episode_files()
for season in metadata.seasons:
if not season.is_spinoff():
continue
tvdb_data = self.tvdb[int(season.tvdb_id)][1]
# Check Length
should = len(episode_files[season.tvdb_id][1])
if not len(tvdb_data) == len(episode_files[season.tvdb_id][1]):
msg = "Mismatch in spinoff {}; Should:{}; Is:{}".format(
season.name, should, len(tvdb_data)
)
valid = self.error(msg)
# Check Names
for episode_number, episode_data in tvdb_data.items():
if not self._has_aired(episode_data):
continue
name = self._generate_episode_name(
season.tvdb_id, 1, episode_number, season.name
)
exists = False
for episode_file in os.listdir(season.path):
if episode_file.startswith(name):
exists = True
if not exists:
valid = self.error(
"Episode {} does not exist "
"or is incorrectly named".format(name)
)
return valid
def _check_anilist_ids(self) -> bool:
"""
Makes sure that every season has at least one anilist ID and that
each anilist ID is entered as "Completed" as long as the series
has already aired
:return: The check result
"""
metadata = self.metadata # type: TvSeries # type: ignore
api = self.config["anilist_api"] # type: AnilistApi
user_list = self.config["anilist_anime_list"]
completed_ids = list(
map(
lambda x: int(x.id.get(AnimeListIdType.ANILIST)),
filter(
lambda x: x.consuming_status == ConsumingStatus.COMPLETED,
user_list
)
)
)
valid = True
for season in metadata.seasons:
mal_ids = None
if IdType.MYANIMELIST not in season.ids:
valid = \
self.error("No myanimelist ID for {}".format(season.name))
else:
mal_ids = season.ids[IdType.MYANIMELIST]
if IdType.ANILIST not in season.ids:
valid = self.error("No anilist ID for {}".format(season.name))
# Fetch IDs based on myanimelist IDs
print(mal_ids)
print(self.fix_interactively)
if mal_ids is not None and self.fix_interactively:
anilist_ids = []
for mal_id in mal_ids:
anilist_id = api.get_anilist_id_from_mal_id(
MediaType.ANIME, int(mal_id)
)
if anilist_id is not None:
anilist_ids.append(str(anilist_id))
resp = input(
"{}Set anilist IDs to {}? (y|n){}".format(
Fore.LIGHTGREEN_EX, anilist_ids, Style.RESET_ALL
)
).lower().strip()
if resp == "y":
season_ids = season.ids
season_ids[IdType.ANILIST] = anilist_ids
season.ids = season_ids
self.metadata.write()
ids = season.ids.get(IdType.ANILIST, [])
for _id in ids:
if int(_id) not in completed_ids:
data = api.get_anime_data(int(_id))
if data.releasing_status == ReleasingStatus.FINISHED:
valid = self.error(
"ID {} not completed on anilist.com".format(_id)
)
return valid
# TODO
# For each ID:
# check if amount of episode files is correct
# Note: Make sure to think of multi-episodes etc
def _generate_ignores_map(self) -> Dict[int, List[int]]:
"""
Generates a dictionary mapping the excluded episode number to their
respective episodes.
:return: The generated dictionary: {season: [episodes]}
"""
metadata = self.metadata # type: TvSeries # type: ignore
ignores = {} # type: Dict[int, List[int]]
excluded = metadata.excludes.get(IdType.TVDB, {})
multis = metadata.multi_episodes.get(IdType.TVDB, {})
start_overrides = \
metadata.season_start_overrides.get(IdType.TVDB, {})
# Add excluded episodes directly
for season, episodes in excluded.items():
ignores[season] = ignores.get(season, []) + episodes
# Add all episodes in a multi episode besides the first one
for season, _multis in multis.items():
for start, end in _multis.items():
ignore = list(range(start + 1, end + 1))
ignores[season] = ignores.get(season, []) + ignore
# Ignore all episodes before the overridden start
for season, start in start_overrides.items():
ignore = list(range(1, start))
ignores[season] = ignores.get(season, []) + ignore
return ignores
def _generate_episode_name(
self,
tvdb_id: str,
season_number: int,
episode_number: int,
series_name_override: Optional[str] = None
):
"""
Generates an episode name
:param tvdb_id: The TVDB ID for which to generate the name
:param season_number: The season number for which to generate the name
:param episode_number: The episode number for which to gen the name
:param series_name_override: Overrides the series name
:return: The generated name
"""
metadata = self.metadata # type: TvSeries # type: ignore
multis = metadata.multi_episodes.get(IdType.TVDB, {})
series_name = metadata.name
if series_name_override is not None:
series_name = series_name_override
end = None
if episode_number in multis.get(season_number, {}):
end = multis[season_number][episode_number]
return RenameOperation.sanitize(
metadata.directory_path,
Renamer.generate_tv_episode_filename(
"",
series_name,
season_number,
episode_number,
Renamer.load_tvdb_episode_name(
tvdb_id,
season_number,
episode_number,
end
),
end
)
)
@staticmethod
def _has_aired(episode_data: dict) -> bool:
"""
Checks whether or not an episode has already aired or not
:param episode_data: The episode data to check
:return: True if already aired, False otherwise
"""
airdate = episode_data["firstAired"]
now = datetime.now().strftime("%Y-%m-%d")
return airdate < now and airdate
toktokkie/check/VisualNovelChecker.py 0 → 100644
View file @ c6cfd4eb
"""
Copyright 2015 Hermann Krumrey <hermann@krumreyh.com>
This file is part of toktokkie.
toktokkie is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
toktokkie is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with toktokkie. If not, see <http://www.gnu.org/licenses/>.
"""
import os
from toktokkie.check.Checker import Checker
from toktokkie.metadata.VisualNovel import VisualNovel
class VisualNovelChecker(Checker):
"""
Class that checks Visual Novels for consistency
"""
def check(self) -> bool:
"""
Performs sanity checks and prints out anything that's wrong
:return: The result of the check
"""
valid = super().check()
valid = self._check_extras() and valid
if self._check_gamefiles():
# Requires game directory to exist
valid = self._check_linux_drm_free_compatible() and valid
else:
valid = False
return valid
def _check_icons(self) -> bool:
"""
Only checks for a main.png icon file.
:return: The result of the check
"""
valid = True
if not os.path.isdir(self.metadata.icon_directory):
valid = self.error("Missing icon directory")
main_icon = os.path.join(self.metadata.icon_directory, "main.png")
if not os.path.isfile(main_icon):
valid = self.error("Missing main icon file for {}".format(
self.metadata.name
))
return valid
def _check_gamefiles(self) -> bool:
"""
Checks if the visual novel has a runscript and game files
:return: The check result
"""
runscript = os.path.join(self.metadata.directory_path, "run.sh")
gamedir = os.path.join(self.metadata.directory_path, "game")
valid = True
if not os.path.isfile(runscript):
valid = self.error("No runscript")
if not os.path.isdir(gamedir):
valid = self.error("No game directory")
return valid
def _check_extras(self) -> bool:
"""
Makes sure that extras, like opening and ending theme videos are
present, unless they've been explicitly excluded using the info.json
file.
:return: The check result
"""
metadata = self.metadata # type: VisualNovel # type: ignore
valid = True
if metadata.has_op and metadata.ops is None:
valid = self.warn("No Opening Video")
if metadata.has_ed and metadata.eds is None:
valid = self.warn("No Ending Video")
if metadata.has_cgs and metadata.cgs is None:
valid = self.warn("No CG Gallery")
if metadata.has_ost and metadata.ost is None:
valid = self.warn("No OST")
return valid
def _check_linux_drm_free_compatible(self) -> bool:
"""
Checks whether or not the game is linux-compatible and DRM free.
This is done by searching for a 'windows', 'steam', 'sony', or
'nintendo' file in the game directory.
If any of those are found, the game is deemed not DRM free and/or
linux compatible
:return: The check result
"""
gamedir = os.path.join(self.metadata.directory_path, "game")
for identifier in ["windows", "steam", "sony", "nintendo"]:
if os.path.isfile(os.path.join(gamedir, identifier)):
return self.warn("Not compatible with linux and DRM free: {}"
.format(identifier))
return True