Nested media index

making the scanner follow symlinks

README.md

@@ -1,4 +1,4 @@
-CouchPotato Server
+CouchPotato
 =====
 
 CouchPotato (CP) is an automatic NZB and torrent downloader. You can keep a "movies I want"-list and it will search for NZBs/torrents of these movies every X hours.
@@ -7,7 +7,7 @@ Once a movie is found, it will send it to SABnzbd or download the torrent to a s
 
 ## Running from Source
 
-CouchPotatoServer can be run from source. This will use *git* as updater, so make sure that is installed also.
+CouchPotatoServer can be run from source. This will use *git* as updater, so make sure that is installed.
 
 Windows, see [the CP forum](http://couchpota.to/forum/showthread.php?tid=14) for more details:
 
@@ -40,7 +40,7 @@ Linux (ubuntu / debian):
 * Make it executable. `sudo chmod +x /etc/init.d/couchpotato`
 * Add it to defaults. `sudo update-rc.d couchpotato defaults`
 * Open your browser and go to: `http://localhost:5050/`
- 
+
 
 FreeBSD :
 
@@ -56,7 +56,7 @@ FreeBSD :
 * Run `git clone https://github.com/RuudBurger/CouchPotatoServer.git`
 * Then run `sudo python CouchPotatoServer/CouchPotato.py` to start for the first time
 * To run on boot copy the init script. `sudo cp CouchPotatoServer/init/freebsd /etc/rc.d/couchpotato`
-* Change the paths inside the init script. `sudo vim /etc/init.d/couchpotato`
+* Change the paths inside the init script. `sudo vim /etc/rc.d/couchpotato`
 * Make init script executable. `sudo chmod +x /etc/rc.d/couchpotato`
 * Add init to startup. `sudo echo 'couchpotato_enable="YES"' >> /etc/rc.conf`
 * Open your browser and go to: `http://server:5050/`

contributing.md

@@ -1,25 +1,36 @@
-## Got a issue/feature request or submitting a pull request?
+# Contributing to CouchPotatoServer
 
-Make sure you think of the following things:
+1. [Contributing](#contributing)
+2. [Submitting an Issue](#issues)
+3. [Submitting a Pull Request](#pull-requests)
 
-## Issue
- * Search through the existing (and closed) issues first, see if you can get your answer there.
- * Double check the result manually, because it could be an external issue. 
- * Post logs! Without seeing what is going on, I can't reproduce the error.
- * Also check the logs before submitting, obvious errors like permission or http errors are often not related to CP.
- * What is the movie + quality you are searching for?
- * What are you're settings for the specific problem?
- * What providers are you using? (While you're logs include these, scanning through hundred of lines of log isn't our hobby)
- * Post the logs from config directory, please do not copy paste the UI. Use pastebin to store these logs!
+## Contributing
+Thank you for your interest in contributing to CouchPotato. There are several ways to help out, even if you've never worked on an open source project before.
+If you've found a bug or want to request a feature, you can report it by [posting an issue](https://github.com/RuudBurger/CouchPotatoServer/issues/new) - be sure to read the [guidelines](#issues) first!
+If you want to contribute your own work, please read the [guidelines](#pull-requests) for submitting a pull request.
+Lastly, for anything related to CouchPotato, feel free to stop by the [forum](http://couchpota.to/forum/) or the [#couchpotato](http://webchat.freenode.net/?channels=couchpotato) IRC channel at irc.freenode.net.
+
+## Issues
+Issues are intended for reporting bugs and weird behaviour or suggesting improvements to CouchPotatoServer.
+Before you submit an issue, please go through the following checklist:
+ * Search through existing issues (*including closed issues!*) first: you might be able to get your answer there.
+ * Double check your issue manually, because it could be an external issue. 
+ * Post logs with your issue: Without seeing what is going on, the developers can't reproduce the error.
+ * Check the logs yourself before submitting them. Obvious errors like permission or HTTP errors are often not related to CouchPotato.
+ * What movie and quality are you searching for?
+ * What are your settings for the specific problem?
+ * What providers are you using? (While your logs include these, scanning through hundreds of lines of logs isn't our hobby)
+ * Post the logs from the *config* directory, please do not copy paste the UI. Use pastebin to store these logs!
  * Give a short step by step of how to reproduce the error.
- * What hardware / OS are you using and what are the limits? NAS can be slow and maybe have a different python installed then when you use CP on OSX or Windows for example.
- * I will mark issues with the "can't reproduce" tag. Don't go asking "why closed" if it clearly says the issue in the tag ;)
- * If you're running on a NAS (QNAP, Austor etc..) with pre-made packages, make sure these are setup to use our source repo (RuudBurger/CouchPotatoServer) and nothing else!!
+ * What hardware / OS are you using and what are its limitations? For example: NAS can be slow and maybe have a different version of python installed then when you use CP on OSX or Windows.
+ * Your issue might be marked with the "can't reproduce" tag. Don't ask why your issue was closed if it says so in the tag.
+ * If you're running on a NAS (QNAP, Austor etc..) with pre-made packages, make sure these are set up to use our source repository (RuudBurger/CouchPotatoServer) and nothing else!!
+ 
+The more relevant information you can provide, the more likely it is the issue will be resolved rather than closed.
 
-## Pull Request
- * Make sure you're pull request is made for develop branch (or relevant feature branch)
+## Pull Requests
+Pull requests are intended for contributing code or documentation to the project. Before you submit a pull request, consider the following:
+ * Make sure your pull request is made for the *develop* branch (or relevant feature branch).
  * Have you tested your PR? If not, why?
- * Are there any limitations of your PR we should know of?
- * Make sure to keep you're PR up-to-date with the branch you're trying to push into.
-
-**If we don't get enough info, the chance of the issue getting closed is a lot bigger ;)**
+ * Does your PR have any limitations we should know of?
+ * Is your PR up-to-date with the branch you're trying to push into?

couchpotato/__init__.py

@@ -15,6 +15,7 @@ log = CPLog(__name__)
 views = {}
 template_loader = template.Loader(os.path.join(os.path.dirname(__file__), 'templates'))
 
+
 class BaseHandler(RequestHandler):
 
     def get_current_user(self):
@@ -48,8 +49,8 @@ def addView(route, func, static = False):
     views[route] = func
 
 
-def get_session():
-    return Env.getSession()
+def get_db():
+    return Env.get('db')
 
 
 # Web view
@@ -71,8 +72,16 @@ def apiDocs():
 addView('docs', apiDocs)
 
 
+# Database debug manager
+def databaseManage():
+    return template_loader.load('database.html').generate(fireEvent = fireEvent, Env = Env)
+
+addView('database', databaseManage)
+
+
 # Make non basic auth option to get api key
 class KeyHandler(RequestHandler):
+
     def get(self, *args, **kwargs):
         api_key = None
 

couchpotato/api.py

@@ -1,15 +1,17 @@
-from couchpotato.core.helpers.request import getParams
-from couchpotato.core.logger import CPLog
 from functools import wraps
 from threading import Thread
-from tornado.gen import coroutine
-from tornado.web import RequestHandler, asynchronous
 import json
 import threading
-import tornado
 import traceback
 import urllib
 
+from couchpotato.core.helpers.request import getParams
+from couchpotato.core.logger import CPLog
+from tornado.gen import coroutine
+from tornado.web import RequestHandler, asynchronous
+import tornado
+
+
 log = CPLog(__name__)
 
 
@@ -93,6 +95,7 @@ class ApiHandler(RequestHandler):
 
             # Split array arguments
             kwargs = getParams(kwargs)
+            kwargs['_request'] = self
 
             # Remove t random string
             try: del kwargs['t']
@@ -127,6 +130,8 @@ class ApiHandler(RequestHandler):
 
         api_locks[route].release()
 
+    post = get
+
 
 def addApiView(route, func, static = False, docs = None, **kwargs):
 

couchpotato/core/_base/_core.py

@@ -1,10 +1,3 @@
-from couchpotato.api import addApiView
-from couchpotato.core.event import fireEvent, addEvent
-from couchpotato.core.helpers.variable import cleanHost, md5
-from couchpotato.core.logger import CPLog
-from couchpotato.core.plugins.base import Plugin
-from couchpotato.environment import Env
-from tornado.ioloop import IOLoop
 from uuid import uuid4
 import os
 import platform
@@ -13,8 +6,19 @@ import time
 import traceback
 import webbrowser
 
+from couchpotato.api import addApiView
+from couchpotato.core.event import fireEvent, addEvent
+from couchpotato.core.helpers.variable import cleanHost, md5
+from couchpotato.core.logger import CPLog
+from couchpotato.core.plugins.base import Plugin
+from couchpotato.environment import Env
+from tornado.ioloop import IOLoop
+
+
 log = CPLog(__name__)
 
+autoload = 'Core'
+
 
 class Core(Plugin):
 
@@ -47,6 +51,7 @@ class Core(Plugin):
         addEvent('app.api_url', self.createApiUrl)
         addEvent('app.version', self.version)
         addEvent('app.load', self.checkDataDir)
+        addEvent('app.load', self.cleanUpFolders)
 
         addEvent('setting.save.core.password', self.md5Password)
         addEvent('setting.save.core.api_key', self.checkApikey)
@@ -71,6 +76,9 @@ class Core(Plugin):
 
         return True
 
+    def cleanUpFolders(self):
+        self.deleteEmptyFolder(Env.get('app_dir'), show_error = False)
+
     def available(self, **kwargs):
         return {
             'success': True
@@ -181,8 +189,104 @@ class Core(Plugin):
     def signalHandler(self):
         if Env.get('daemonized'): return
 
-        def signal_handler(signal, frame):
+        def signal_handler(*args, **kwargs):
             fireEvent('app.shutdown', single = True)
 
         signal.signal(signal.SIGINT, signal_handler)
         signal.signal(signal.SIGTERM, signal_handler)
+
+
+config = [{
+    'name': 'core',
+    'order': 1,
+    'groups': [
+        {
+            'tab': 'general',
+            'name': 'basics',
+            'description': 'Needs restart before changes take effect.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'port',
+                    'default': 5050,
+                    'type': 'int',
+                    'description': 'The port I should listen to.',
+                },
+                {
+                    'name': 'ssl_cert',
+                    'description': 'Path to SSL server.crt',
+                    'advanced': True,
+                },
+                {
+                    'name': 'ssl_key',
+                    'description': 'Path to SSL server.key',
+                    'advanced': True,
+                },
+                {
+                    'name': 'launch_browser',
+                    'default': True,
+                    'type': 'bool',
+                    'description': 'Launch the browser when I start.',
+                    'wizard': True,
+                },
+            ],
+        },
+        {
+            'tab': 'general',
+            'name': 'advanced',
+            'description': "For those who know what they're doing",
+            'advanced': True,
+            'options': [
+                {
+                    'name': 'api_key',
+                    'default': uuid4().hex,
+                    'readonly': 1,
+                    'description': 'Let 3rd party app do stuff. <a target="_self" href="../../docs/">Docs</a>',
+                },
+                {
+                    'name': 'debug',
+                    'default': 0,
+                    'type': 'bool',
+                    'description': 'Enable debugging.',
+                },
+                {
+                    'name': 'development',
+                    'default': 0,
+                    'type': 'bool',
+                    'description': 'Enable this if you\'re developing, and NOT in any other case, thanks.',
+                },
+                {
+                    'name': 'data_dir',
+                    'type': 'directory',
+                    'description': 'Where cache/logs/etc are stored. Keep empty for defaults.',
+                },
+                {
+                    'name': 'url_base',
+                    'default': '',
+                    'description': 'When using mod_proxy use this to append the url with this.',
+                },
+                {
+                    'name': 'permission_folder',
+                    'default': '0755',
+                    'label': 'Folder CHMOD',
+                    'description': 'Can be either decimal (493) or octal (leading zero: 0755)',
+                },
+                {
+                    'name': 'permission_file',
+                    'default': '0755',
+                    'label': 'File CHMOD',
+                    'description': 'Same as Folder CHMOD but for files',
+                },
+            ],
+        },
+    ],
+}]

couchpotato/core/_base/_core/__init__.py

@@ -1,101 +0,0 @@
-from .main import Core
-from uuid import uuid4
-
-
-def start():
-    return Core()
-
-config = [{
-    'name': 'core',
-    'order': 1,
-    'groups': [
-        {
-            'tab': 'general',
-            'name': 'basics',
-            'description': 'Needs restart before changes take effect.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'username',
-                    'default': '',
-                },
-                {
-                    'name': 'password',
-                    'default': '',
-                    'type': 'password',
-                },
-                {
-                    'name': 'port',
-                    'default': 5050,
-                    'type': 'int',
-                    'description': 'The port I should listen to.',
-                },
-                {
-                    'name': 'ssl_cert',
-                    'description': 'Path to SSL server.crt',
-                    'advanced': True,
-                },
-                {
-                    'name': 'ssl_key',
-                    'description': 'Path to SSL server.key',
-                    'advanced': True,
-                },
-                {
-                    'name': 'launch_browser',
-                    'default': True,
-                    'type': 'bool',
-                    'description': 'Launch the browser when I start.',
-                    'wizard': True,
-                },
-            ],
-        },
-        {
-            'tab': 'general',
-            'name': 'advanced',
-            'description': "For those who know what they're doing",
-            'advanced': True,
-            'options': [
-                {
-                    'name': 'api_key',
-                    'default': uuid4().hex,
-                    'readonly': 1,
-                    'description': 'Let 3rd party app do stuff. <a target="_self" href="../../docs/">Docs</a>',
-                },
-                {
-                    'name': 'debug',
-                    'default': 0,
-                    'type': 'bool',
-                    'description': 'Enable debugging.',
-                },
-                {
-                    'name': 'development',
-                    'default': 0,
-                    'type': 'bool',
-                    'description': 'Enable this if you\'re developing, and NOT in any other case, thanks.',
-                },
-                {
-                    'name': 'data_dir',
-                    'type': 'directory',
-                    'description': 'Where cache/logs/etc are stored. Keep empty for defaults.',
-                },
-                {
-                    'name': 'url_base',
-                    'default': '',
-                    'description': 'When using mod_proxy use this to append the url with this.',
-                },
-                {
-                    'name': 'permission_folder',
-                    'default': '0755',
-                    'label': 'Folder CHMOD',
-                    'description': 'Can be either decimal (493) or octal (leading zero: 0755)',
-                },
-                {
-                    'name': 'permission_file',
-                    'default': '0755',
-                    'label': 'File CHMOD',
-                    'description': 'Same as Folder CHMOD but for files',
-                },
-            ],
-        },
-    ],
-}]

couchpotato/core/_base/clientscript.py

@@ -1,3 +1,7 @@
+import os
+import re
+import traceback
+
 from couchpotato.core.event import addEvent
 from couchpotato.core.helpers.encoding import ss
 from couchpotato.core.helpers.variable import tryInt
@@ -7,12 +11,12 @@ from couchpotato.environment import Env
 from minify.cssmin import cssmin
 from minify.jsmin import jsmin
 from tornado.web import StaticFileHandler
-import os
-import re
-import traceback
+
 
 log = CPLog(__name__)
 
+autoload = 'ClientScript'
+
 
 class ClientScript(Plugin):
 
@@ -45,21 +49,17 @@ class ClientScript(Plugin):
             'scripts/block/footer.js',
             'scripts/block/menu.js',
             'scripts/page/home.js',
-            'scripts/page/wanted.js',
             'scripts/page/settings.js',
             'scripts/page/about.js',
-            'scripts/page/manage.js',
-            'scripts/misc/downloaders.js',
         ],
     }
 
-
     urls = {'style': {}, 'script': {}}
     minified = {'style': {}, 'script': {}}
     paths = {'style': {}, 'script': {}}
     comment = {
-       'style': '/*** %s:%d ***/\n',
-       'script': '// %s:%d\n'
+        'style': '/*** %s:%d ***/\n',
+        'script': '// %s:%d\n'
     }
 
     html = {
@@ -91,7 +91,6 @@ class ClientScript(Plugin):
                 else:
                     self.registerStyle(core_url, file_path, position = 'front')
 
-
     def minify(self):
 
         # Create cache dir
@@ -125,7 +124,7 @@ class ClientScript(Plugin):
                 data = cssmin(data)
                 data = data.replace('../images/', '../static/images/')
                 data = data.replace('../fonts/', '../static/fonts/')
-                data = data.replace('../../static/', '../static/') # Replace inside plugins
+                data = data.replace('../../static/', '../static/')  # Replace inside plugins
 
             raw.append({'file': file_path, 'date': int(os.path.getmtime(file_path)), 'data': data})
 
@@ -188,6 +187,7 @@ class ClientScript(Plugin):
 
     prefix_properties = ['border-radius', 'transform', 'transition', 'box-shadow']
     prefix_tags = ['ms', 'moz', 'webkit']
+
     def prefix(self, data):
 
         trimmed_data = re.sub('(\t|\n|\r)+', '', data)

couchpotato/core/_base/clientscript/__init__.py

@@ -1,7 +0,0 @@
-from .main import ClientScript
-
-
-def start():
-    return ClientScript()
-
-config = []

couchpotato/core/_base/desktop.py

@@ -5,6 +5,9 @@ from couchpotato.environment import Env
 
 log = CPLog(__name__)
 
+autoload = 'Desktop'
+
+
 if Env.get('desktop'):
 
     class Desktop(Plugin):

couchpotato/core/_base/desktop/__init__.py

@@ -1,7 +0,0 @@
-from .main import Desktop
-
-
-def start():
-    return Desktop()
-
-config = []

couchpotato/core/_base/downloader/__init__.py

@@ -0,0 +1,20 @@
+from .main import Downloader
+
+
+def autoload():
+    return Downloader()
+
+
+config = [{
+    'name': 'download_providers',
+    'groups': [
+        {
+            'label': 'Downloaders',
+            'description': 'You can select different downloaders for each type (usenet / torrent)',
+            'type': 'list',
+            'name': 'download_providers',
+            'tab': 'downloaders',
+            'options': [],
+        },
+    ],
+}]

couchpotato/core/_base/downloader/main.py

@@ -1,16 +1,24 @@
 from base64 import b32decode, b16encode
+import random
+import re
+
 from couchpotato.api import addApiView
 from couchpotato.core.event import addEvent
 from couchpotato.core.helpers.variable import mergeDicts
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.base import Provider
-import random
-import re
+from couchpotato.core.media._base.providers.base import Provider
+from couchpotato.core.plugins.base import Plugin
+
 
 log = CPLog(__name__)
 
 
-class Downloader(Provider):
+## This is here to load the static files
+class Downloader(Plugin):
+    pass
+
+
+class DownloaderBase(Provider):
 
     protocol = []
     http_time_between_calls = 0
@@ -22,17 +30,21 @@ class Downloader(Provider):
     ]
 
     torrent_trackers = [
-        'http://tracker.publicbt.com/announce',
         'udp://tracker.istole.it:80/announce',
-        'udp://fr33domtracker.h33t.com:3310/announce',
         'http://tracker.istole.it/announce',
-        'http://tracker.ccc.de/announce',
+        'udp://fr33domtracker.h33t.com:3310/announce',
+        'http://tracker.publicbt.com/announce',
         'udp://tracker.publicbt.com:80/announce',
+        'http://tracker.ccc.de/announce',
         'udp://tracker.ccc.de:80/announce',
         'http://exodus.desync.com/announce',
         'http://exodus.desync.com:6969/announce',
         'http://tracker.publichd.eu/announce',
+        'udp://tracker.publichd.eu:80/announce',
         'http://tracker.openbittorrent.com/announce',
+        'udp://tracker.openbittorrent.com/announce',
+        'udp://tracker.openbittorrent.com:80/announce',
+        'udp://open.demonii.com:1337/announce',
     ]
 
     def __init__(self):
@@ -156,11 +168,11 @@ class Downloader(Provider):
         if not data: data = {}
 
         d_manual = self.conf('manual', default = False)
-        return super(Downloader, self).isEnabled() and \
+        return super(DownloaderBase, self).isEnabled() and \
             (d_manual and manual or d_manual is False) and \
             (not data or self.isCorrectProtocol(data.get('protocol')))
 
-    def _test(self):
+    def _test(self, **kwargs):
         t = self.test()
         if isinstance(t, tuple):
             return {'success': t[0], 'msg': t[1]}
@@ -182,6 +194,7 @@ class Downloader(Provider):
     def pause(self, release_download, pause):
         return
 
+
 class ReleaseDownloadList(list):
 
     provider = None
@@ -208,7 +221,7 @@ class ReleaseDownloadList(list):
             'status': 'busy',
             'downloader': self.provider.getName(),
             'folder': '',
-            'files': '',
+            'files': [],
         }
 
         return mergeDicts(defaults, result)

couchpotato/core/_base/downloader/static/downloaders.js

@@ -6,7 +6,7 @@ var DownloadersBase = new Class({
 		var self = this;
 
 		// Add test buttons to settings page
-		App.addEvent('load', self.addTestButtons.bind(self));
+		App.addEvent('loadSettings', self.addTestButtons.bind(self));
 
 	},
 
@@ -68,8 +68,8 @@ var DownloadersBase = new Class({
 	testButtonName: function(fieldset){
 		var name = String(fieldset.getElement('h2').innerHTML).substring(0,String(fieldset.getElement('h2').innerHTML).indexOf("<span"));
 		return 'Test '+name;
-	},
+	}
 
 });
 
-window.Downloaders = new DownloadersBase();
+var Downloaders = new DownloadersBase();

couchpotato/core/_base/scheduler.py

@@ -5,6 +5,8 @@ from couchpotato.core.plugins.base import Plugin
 
 log = CPLog(__name__)
 
+autoload = 'Scheduler'
+
 
 class Scheduler(Plugin):
 
@@ -66,6 +68,8 @@ class Scheduler(Plugin):
             'job': self.sched.add_interval_job(handle, hours = hours, minutes = minutes, seconds = seconds)
         }
 
+        return True
+
     def queue(self, handlers = None):
         if not handlers: handlers = []
 

couchpotato/core/_base/scheduler/__init__.py

@@ -1,7 +0,0 @@
-from .main import Scheduler
-
-
-def start():
-    return Scheduler()
-
-config = []

couchpotato/core/_base/updater/__init__.py

@@ -1,9 +1,10 @@
-from .main import Updater
-from couchpotato.environment import Env
 import os
 
+from .main import Updater
+from couchpotato.environment import Env
 
-def start():
+
+def autoload():
     return Updater()
 
 config = [{

couchpotato/core/_base/updater/main.py

@@ -1,28 +1,33 @@
-from couchpotato.api import addApiView
-from couchpotato.core.event import addEvent, fireEvent, fireEventAsync
-from couchpotato.core.helpers.encoding import ss
-from couchpotato.core.logger import CPLog
-from couchpotato.core.plugins.base import Plugin
-from couchpotato.environment import Env
-from datetime import datetime
-from dateutil.parser import parse
-from git.repository import LocalRepository
 import json
 import os
 import shutil
 import tarfile
 import time
 import traceback
-import version
 import zipfile
+from datetime import datetime
+from threading import RLock
+
+from couchpotato.api import addApiView
+from couchpotato.core.event import addEvent, fireEvent, fireEventAsync
+from couchpotato.core.helpers.encoding import ss
+from couchpotato.core.logger import CPLog
+from couchpotato.core.plugins.base import Plugin
+from couchpotato.environment import Env
+from dateutil.parser import parse
+from git.repository import LocalRepository
+from scandir import scandir
+import version
 from six.moves import filter
 
+
 log = CPLog(__name__)
 
 
 class Updater(Plugin):
 
     available_notified = False
+    _lock = RLock()
 
     def __init__(self):
 
@@ -100,7 +105,17 @@ class Updater(Plugin):
         return False
 
     def info(self, **kwargs):
-        return self.updater.info()
+        self._lock.acquire()
+
+        info = {}
+        try:
+            info = self.updater.info()
+        except:
+            log.error('Failed getting updater info: %s', traceback.format_exc())
+
+        self._lock.release()
+
+        return info
 
     def checkView(self, **kwargs):
         return {
@@ -127,6 +142,10 @@ class Updater(Plugin):
             'success': success
         }
 
+    def doShutdown(self):
+        self.updater.deletePyc(show_logs = False)
+        return super(Updater, self).doShutdown()
+
 
 class BaseUpdater(Plugin):
 
@@ -144,12 +163,15 @@ class BaseUpdater(Plugin):
         pass
 
     def info(self):
+
+        current_version = self.getVersion()
+
         return {
             'last_check': self.last_check,
             'update_version': self.update_version,
-            'version': self.getVersion(),
+            'version': current_version,
             'repo_name': '%s/%s' % (self.repo_user, self.repo_name),
-            'branch': self.branch,
+            'branch': current_version.get('branch', self.branch),
         }
 
     def getVersion(self):
@@ -158,9 +180,9 @@ class BaseUpdater(Plugin):
     def check(self):
         pass
 
-    def deletePyc(self, only_excess = True):
+    def deletePyc(self, only_excess = True, show_logs = True):
 
-        for root, dirs, files in os.walk(ss(Env.get('app_dir'))):
+        for root, dirs, files in scandir.walk(ss(Env.get('app_dir'))):
 
             pyc_files = filter(lambda filename: filename.endswith('.pyc'), files)
             py_files = set(filter(lambda filename: filename.endswith('.py'), files))
@@ -168,7 +190,7 @@ class BaseUpdater(Plugin):
 
             for excess_pyc_file in excess_pyc_files:
                 full_path = os.path.join(root, excess_pyc_file)
-                log.debug('Removing old PYC file: %s', full_path)
+                if show_logs: log.debug('Removing old PYC file: %s', full_path)
                 try:
                     os.remove(full_path)
                 except:
@@ -194,9 +216,6 @@ class GitUpdater(BaseUpdater):
             log.info('Updating to latest version')
             self.repo.pull()
 
-            # Delete leftover .pyc files
-            self.deletePyc()
-
             return True
         except:
             log.error('Failed updating via GIT: %s', traceback.format_exc())
@@ -212,10 +231,11 @@ class GitUpdater(BaseUpdater):
                 output = self.repo.getHead()  # Yes, please
                 log.debug('Git version output: %s', output.hash)
                 self.version = {
-                    'repr': 'git:(%s:%s % s) %s (%s)' % (self.repo_user, self.repo_name, self.branch, output.hash[:8], datetime.fromtimestamp(output.getDate())),
+                    'repr': 'git:(%s:%s % s) %s (%s)' % (self.repo_user, self.repo_name, self.repo.getCurrentBranch().name or self.branch, output.hash[:8], datetime.fromtimestamp(output.getDate())),
                     'hash': output.hash[:8],
                     'date': output.getDate(),
                     'type': 'git',
+                    'branch': self.repo.getCurrentBranch().name
                 }
             except Exception as e:
                 log.error('Failed using GIT updater, running from source, you need to have GIT installed. %s', e)
@@ -308,11 +328,11 @@ class SourceUpdater(BaseUpdater):
         # Get list of files we want to overwrite
         self.deletePyc()
         existing_files = []
-        for root, subfiles, filenames in os.walk(app_dir):
+        for root, subfiles, filenames in scandir.walk(app_dir):
             for filename in filenames:
                 existing_files.append(os.path.join(root, filename))
 
-        for root, subfiles, filenames in os.walk(path):
+        for root, subfiles, filenames in scandir.walk(path):
             for filename in filenames:
                 fromfile = os.path.join(root, filename)
                 tofile = os.path.join(app_dir, fromfile.replace(path + os.path.sep, ''))

couchpotato/core/_base/updater/static/updater.js

@@ -5,7 +5,7 @@ var UpdaterBase = new Class({
 	initialize: function(){
 		var self = this;
 
-		App.addEvent('load', self.info.bind(self, 2000))
+		App.addEvent('load', self.info.bind(self, 2000));
 		App.addEvent('unload', function(){
 			if(self.timer)
 				clearTimeout(self.timer);
@@ -66,7 +66,7 @@ var UpdaterBase = new Class({
 
 		var changelog = 'https://github.com/'+data.repo_name+'/compare/'+data.version.hash+'...'+data.branch;
 		if(data.update_version.changelog)
-			changelog = data.update_version.changelog + '#' + data.version.hash+'...'+data.update_version.hash
+			changelog = data.update_version.changelog + '#' + data.version.hash+'...'+data.update_version.hash;
 
 		self.message = new Element('div.message.update').adopt(
 			new Element('span', {

couchpotato/core/database.py

@@ -0,0 +1,482 @@
+import json
+import os
+import time
+import traceback
+
+from couchpotato import CPLog
+from couchpotato.api import addApiView
+from couchpotato.core.event import addEvent, fireEvent
+from couchpotato.core.helpers.encoding import toUnicode
+from couchpotato.core.helpers.variable import getImdb, tryInt
+
+
+log = CPLog(__name__)
+
+
+class Database(object):
+
+    indexes = []
+    db = None
+
+    def __init__(self):
+
+        addApiView('database.list_documents', self.listDocuments)
+        addApiView('database.reindex', self.reindex)
+        addApiView('database.compact', self.compact)
+        addApiView('database.document.update', self.updateDocument)
+        addApiView('database.document.delete', self.deleteDocument)
+
+        addEvent('database.setup_index', self.setupIndex)
+        addEvent('app.migrate', self.migrate)
+
+    def getDB(self):
+
+        if not self.db:
+            from couchpotato import get_db
+            self.db = get_db()
+
+        return self.db
+
+    def setupIndex(self, index_name, klass):
+
+        self.indexes.append(index_name)
+
+        db = self.getDB()
+
+        # Category index
+        index_instance = klass(db.path, index_name)
+        try:
+            db.add_index(index_instance)
+            db.reindex_index(index_name)
+        except:
+            previous = db.indexes_names[index_name]
+            previous_version = previous._version
+            current_version = klass._version
+
+            # Only edit index if versions are different
+            if previous_version < current_version:
+                log.debug('Index "%s" already exists, updating and reindexing', index_name)
+                db.destroy_index(previous)
+                db.add_index(index_instance)
+                db.reindex_index(index_name)
+
+    def deleteDocument(self, **kwargs):
+
+        db = self.getDB()
+
+        try:
+
+            document_id = kwargs.get('_request').get_argument('id')
+            document = db.get('id', document_id)
+            db.delete(document)
+
+            return {
+                'success': True
+            }
+        except:
+            return {
+                'success': False,
+                'error': traceback.format_exc()
+            }
+
+    def updateDocument(self, **kwargs):
+
+        db = self.getDB()
+
+        try:
+
+            document = json.loads(kwargs.get('_request').get_argument('document'))
+            d = db.update(document)
+            document.update(d)
+
+            return {
+                'success': True,
+                'document': document
+            }
+        except:
+            return {
+                'success': False,
+                'error': traceback.format_exc()
+            }
+
+    def listDocuments(self, **kwargs):
+        db = self.getDB()
+
+        results = {
+            'unknown': []
+        }
+
+        for document in db.all('id'):
+            key = document.get('_t', 'unknown')
+
+            if kwargs.get('show') and key != kwargs.get('show'):
+                continue
+
+            if not results.get(key):
+                results[key] = []
+            results[key].append(document)
+
+        return results
+
+    def reindex(self, **kwargs):
+
+        success = True
+        try:
+            db = self.getDB()
+            db.reindex()
+        except:
+            log.error('Failed index: %s', traceback.format_exc())
+            success = False
+
+        return {
+            'success': success
+        }
+
+    def compact(self, **kwargs):
+
+        success = True
+        try:
+            db = self.getDB()
+            db.compact()
+        except:
+            log.error('Failed compact: %s', traceback.format_exc())
+            success = False
+
+        return {
+            'success': success
+        }
+
+    def migrate(self):
+
+        from couchpotato import Env
+        old_db = os.path.join(Env.get('data_dir'), 'couchpotato.db')
+        if not os.path.isfile(old_db): return
+
+        log.info('=' * 30)
+        log.info('Migrating database, hold on..')
+        time.sleep(1)
+
+        if os.path.isfile(old_db):
+
+            migrate_start = time.time()
+
+            import sqlite3
+            conn = sqlite3.connect(old_db)
+
+            migrate_list = {
+                'category': ['id', 'label', 'order', 'required', 'preferred', 'ignored', 'destination'],
+                'profile': ['id', 'label', 'order', 'core', 'hide'],
+                'profiletype': ['id', 'order', 'finish', 'wait_for', 'quality_id', 'profile_id'],
+                'quality': ['id', 'identifier', 'order', 'size_min', 'size_max'],
+                'movie': ['id', 'last_edit', 'library_id', 'status_id', 'profile_id', 'category_id'],
+                'library': ['id', 'identifier', 'info'],
+                'librarytitle': ['id', 'title', 'default', 'libraries_id'],
+                'library_files__file_library': ['library_id', 'file_id'],
+                'release': ['id', 'identifier', 'movie_id', 'status_id', 'quality_id', 'last_edit'],
+                'releaseinfo': ['id', 'identifier', 'value', 'release_id'],
+                'release_files__file_release': ['release_id', 'file_id'],
+                'status': ['id', 'identifier'],
+                'properties': ['id', 'identifier', 'value'],
+                'file': ['id', 'path', 'type_id'],
+                'filetype': ['identifier', 'id']
+            }
+
+            migrate_data = {}
+
+            c = conn.cursor()
+
+            for ml in migrate_list:
+                migrate_data[ml] = {}
+                rows = migrate_list[ml]
+
+                try:
+                    c.execute('SELECT %s FROM `%s`' % ('`' + '`,`'.join(rows) + '`', ml))
+                except:
+                    # ignore faulty destination_id database
+                    if ml == 'category':
+                        migrate_data[ml] = {}
+                    else:
+                        raise
+
+                for p in c.fetchall():
+                    columns = {}
+                    for row in migrate_list[ml]:
+                        columns[row] = p[rows.index(row)]
+
+                    if not migrate_data[ml].get(p[0]):
+                        migrate_data[ml][p[0]] = columns
+                    else:
+                        if not isinstance(migrate_data[ml][p[0]], list):
+                            migrate_data[ml][p[0]] = [migrate_data[ml][p[0]]]
+                        migrate_data[ml][p[0]].append(columns)
+
+            conn.close()
+
+            log.info('Getting data took %s', time.time() - migrate_start)
+
+            db = self.getDB()
+
+            # Use properties
+            properties = migrate_data['properties']
+            log.info('Importing %s properties', len(properties))
+            for x in properties:
+                property = properties[x]
+                Env.prop(property.get('identifier'), property.get('value'))
+
+            # Categories
+            categories = migrate_data.get('category', [])
+            log.info('Importing %s categories', len(categories))
+            category_link = {}
+            for x in categories:
+                c = categories[x]
+
+                new_c = db.insert({
+                    '_t': 'category',
+                    'order': c.get('order', 999),
+                    'label': toUnicode(c.get('label', '')),
+                    'ignored': toUnicode(c.get('ignored', '')),
+                    'preferred': toUnicode(c.get('preferred', '')),
+                    'required': toUnicode(c.get('required', '')),
+                    'destination': toUnicode(c.get('destination', '')),
+                })
+
+                category_link[x] = new_c.get('_id')
+
+            # Profiles
+            log.info('Importing profiles')
+            new_profiles = db.all('profile', with_doc = True)
+            new_profiles_by_label = {}
+            for x in new_profiles:
+
+                # Remove default non core profiles
+                if not x['doc'].get('core'):
+                    db.delete(x['doc'])
+                else:
+                    new_profiles_by_label[x['doc']['label']] = x['_id']
+
+            profiles = migrate_data['profile']
+            profile_link = {}
+            for x in profiles:
+                p = profiles[x]
+
+                exists = new_profiles_by_label.get(p.get('label'))
+
+                # Update existing with order only
+                if exists and p.get('core'):
+                    profile = db.get('id', exists)
+                    profile['order'] = tryInt(p.get('order'))
+                    profile['hide'] = p.get('hide') in [1, True, 'true', 'True']
+                    db.update(profile)
+
+                    profile_link[x] = profile.get('_id')
+                else:
+
+                    new_profile = {
+                        '_t': 'profile',
+                        'label': p.get('label'),
+                        'order': int(p.get('order', 999)),
+                        'core': p.get('core', False),
+                        'qualities': [],
+                        'wait_for': [],
+                        'finish': []
+                    }
+
+                    types = migrate_data['profiletype']
+                    for profile_type in types:
+                        p_type = types[profile_type]
+                        if types[profile_type]['profile_id'] == p['id']:
+                            new_profile['finish'].append(p_type['finish'])
+                            new_profile['wait_for'].append(p_type['wait_for'])
+                            new_profile['qualities'].append(migrate_data['quality'][p_type['quality_id']]['identifier'])
+
+                    new_profile.update(db.insert(new_profile))
+
+                    profile_link[x] = new_profile.get('_id')
+
+            # Qualities
+            log.info('Importing quality sizes')
+            new_qualities = db.all('quality', with_doc = True)
+            new_qualities_by_identifier = {}
+            for x in new_qualities:
+                new_qualities_by_identifier[x['doc']['identifier']] = x['_id']
+
+            qualities = migrate_data['quality']
+            quality_link = {}
+            for x in qualities:
+                q = qualities[x]
+                q_id = new_qualities_by_identifier[q.get('identifier')]
+
+                quality = db.get('id', q_id)
+                quality['order'] = q.get('order')
+                quality['size_min'] = tryInt(q.get('size_min'))
+                quality['size_max'] = tryInt(q.get('size_max'))
+                db.update(quality)
+
+                quality_link[x] = quality
+
+            # Titles
+            titles = migrate_data['librarytitle']
+            titles_by_library = {}
+            for x in titles:
+                title = titles[x]
+                if title.get('default'):
+                    titles_by_library[title.get('libraries_id')] = title.get('title')
+
+            # Releases
+            releaseinfos = migrate_data['releaseinfo']
+            for x in releaseinfos:
+                info = releaseinfos[x]
+
+                # Skip if release doesn't exist for this info
+                if not migrate_data['release'].get(info.get('release_id')):
+                    continue
+
+                if not migrate_data['release'][info.get('release_id')].get('info'):
+                    migrate_data['release'][info.get('release_id')]['info'] = {}
+
+                migrate_data['release'][info.get('release_id')]['info'][info.get('identifier')] = info.get('value')
+
+            releases = migrate_data['release']
+            releases_by_media = {}
+            for x in releases:
+                release = releases[x]
+                if not releases_by_media.get(release.get('movie_id')):
+                    releases_by_media[release.get('movie_id')] = []
+
+                releases_by_media[release.get('movie_id')].append(release)
+
+            # Type ids
+            types = migrate_data['filetype']
+            type_by_id = {}
+            for t in types:
+                type = types[t]
+                type_by_id[type.get('id')] = type
+
+            # Media
+            log.info('Importing %s media items', len(migrate_data['movie']))
+            statuses = migrate_data['status']
+            libraries = migrate_data['library']
+            library_files = migrate_data['library_files__file_library']
+            releases_files = migrate_data['release_files__file_release']
+            all_files = migrate_data['file']
+            poster_type = migrate_data['filetype']['poster']
+            medias = migrate_data['movie']
+            for x in medias:
+                m = medias[x]
+
+                status = statuses.get(m['status_id']).get('identifier')
+                l = libraries[m['library_id']]
+
+                # Only migrate wanted movies, Skip if no identifier present
+                if not getImdb(l.get('identifier')): continue
+
+                profile_id = profile_link.get(m['profile_id'])
+                category_id = category_link.get(m['category_id'])
+                title = titles_by_library.get(m['library_id'])
+                releases = releases_by_media.get(x, [])
+                info = json.loads(l.get('info', ''))
+
+                files = library_files.get(m['library_id'], [])
+                if not isinstance(files, list):
+                    files = [files]
+
+                added_media = fireEvent('movie.add', {
+                    'info': info,
+                    'identifier': l.get('identifier'),
+                    'profile_id': profile_id,
+                    'category_id': category_id,
+                    'title': title
+                }, force_readd = False, search_after = False, update_after = False, notify_after = False, status = status, single = True)
+
+                if not added_media:
+                    log.error('Failed adding media %s: %s', (l.get('identifier'), info))
+                    continue
+
+                added_media['files'] = added_media.get('files', {})
+                for f in files:
+                    ffile = all_files[f.get('file_id')]
+
+                    # Only migrate posters
+                    if ffile.get('type_id') == poster_type.get('id'):
+                        if ffile.get('path') not in added_media['files'].get('image_poster', []) and os.path.isfile(ffile.get('path')):
+                            added_media['files']['image_poster'] = [ffile.get('path')]
+                            break
+
+                if 'image_poster' in added_media['files']:
+                    db.update(added_media)
+
+                for rel in releases:
+
+                    empty_info = False
+                    if not rel.get('info'):
+                        empty_info = True
+                        rel['info'] = {}
+
+                    quality = quality_link[rel.get('quality_id')]
+                    release_status = statuses.get(rel.get('status_id')).get('identifier')
+
+                    if rel['info'].get('download_id'):
+                        status_support = rel['info'].get('download_status_support', False) in [True, 'true', 'True']
+                        rel['info']['download_info'] = {
+                            'id': rel['info'].get('download_id'),
+                            'downloader': rel['info'].get('download_downloader'),
+                            'status_support': status_support,
+                        }
+
+                    # Add status to keys
+                    rel['info']['status'] = release_status
+                    if not empty_info:
+                        fireEvent('release.create_from_search', [rel['info']], added_media, quality, single = True)
+                    else:
+                        release = {
+                            '_t': 'release',
+                            'identifier': rel.get('identifier'),
+                            'media_id': added_media.get('_id'),
+                            'quality': quality.get('identifier'),
+                            'status': release_status,
+                            'last_edit': int(time.time()),
+                            'files': {}
+                        }
+
+                        # Add downloader info if provided
+                        try:
+                            release['download_info'] = rel['info']['download_info']
+                            del rel['download_info']
+                        except:
+                            pass
+
+                        # Add files
+                        release_files = releases_files.get(rel.get('id'), [])
+                        if not isinstance(release_files, list):
+                            release_files = [release_files]
+
+                        if len(release_files) == 0:
+                            continue
+
+                        for f in release_files:
+                            rfile = all_files[f.get('file_id')]
+                            file_type = type_by_id.get(rfile.get('type_id')).get('identifier')
+
+                            if not release['files'].get(file_type):
+                                release['files'][file_type] = []
+
+                            release['files'][file_type].append(rfile.get('path'))
+
+                        try:
+                            rls = db.get('release_identifier', rel.get('identifier'), with_doc = True)['doc']
+                            rls.update(release)
+                            db.update(rls)
+                        except:
+                            db.insert(release)
+
+            log.info('Total migration took %s', time.time() - migrate_start)
+            log.info('=' * 30)
+
+            # rename old database
+            log.info('Renaming old database to %s ', old_db + '.old')
+            os.rename(old_db, old_db + '.old')
+
+            if os.path.isfile(old_db + '-wal'):
+                os.rename(old_db + '-wal', old_db + '-wal.old')
+            if os.path.isfile(old_db + '-shm'):
+                os.rename(old_db + '-shm', old_db + '-shm.old')

couchpotato/core/downloaders/__init__.py

@@ -1,13 +0,0 @@
-config = [{
-    'name': 'download_providers',
-    'groups': [
-        {
-            'label': 'Downloaders',
-            'description': 'You can select different downloaders for each type (usenet / torrent)',
-            'type': 'list',
-            'name': 'download_providers',
-            'tab': 'downloaders',
-            'options': [],
-        },
-    ],
-}]

couchpotato/core/downloaders/blackhole.py

@@ -1,15 +1,20 @@
 from __future__ import with_statement
-from couchpotato.core.downloaders.base import Downloader
-from couchpotato.core.helpers.encoding import sp
-from couchpotato.core.logger import CPLog
-from couchpotato.environment import Env
 import os
 import traceback
 
+from couchpotato.core._base.downloader.main import DownloaderBase
+from couchpotato.core.helpers.encoding import sp
+from couchpotato.core.helpers.variable import getDownloadDir
+from couchpotato.core.logger import CPLog
+from couchpotato.environment import Env
+
+
 log = CPLog(__name__)
 
+autoload = 'Blackhole'
 
-class Blackhole(Downloader):
+
+class Blackhole(DownloaderBase):
 
     protocol = ['nzb', 'torrent', 'torrent_magnet']
     status_support = False
@@ -100,3 +105,54 @@ class Blackhole(Downloader):
 
         return super(Blackhole, self).isEnabled(manual, data) and \
             ((self.conf('use_for') in for_protocol))
+
+
+config = [{
+    'name': 'blackhole',
+    'order': 30,
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'blackhole',
+            'label': 'Black hole',
+            'description': 'Download the NZB/Torrent to a specific folder. <em>Note: Seeding and copying/linking features do <strong>not</strong> work with Black hole</em>.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': True,
+                    'type': 'enabler',
+                    'radio_group': 'nzb,torrent',
+                },
+                {
+                    'name': 'directory',
+                    'type': 'directory',
+                    'description': 'Directory where the .nzb (or .torrent) file is saved to.',
+                    'default': getDownloadDir()
+                },
+                {
+                    'name': 'use_for',
+                    'label': 'Use for',
+                    'default': 'both',
+                    'type': 'dropdown',
+                    'values': [('usenet & torrents', 'both'), ('usenet', 'nzb'), ('torrent', 'torrent')],
+                },
+                {
+                    'name': 'create_subdir',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Create a sub directory when saving the .nzb (or .torrent).',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/blackhole/__init__.py

@@ -1,56 +0,0 @@
-from .main import Blackhole
-from couchpotato.core.helpers.variable import getDownloadDir
-
-
-def start():
-    return Blackhole()
-
-config = [{
-    'name': 'blackhole',
-    'order': 30,
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'blackhole',
-            'label': 'Black hole',
-            'description': 'Download the NZB/Torrent to a specific folder. <em>Note: Seeding and copying/linking features do <strong>not</strong> work with Black hole</em>.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': True,
-                    'type': 'enabler',
-                    'radio_group': 'nzb,torrent',
-                },
-                {
-                    'name': 'directory',
-                    'type': 'directory',
-                    'description': 'Directory where the .nzb (or .torrent) file is saved to.',
-                    'default': getDownloadDir()
-                },
-                {
-                    'name': 'use_for',
-                    'label': 'Use for',
-                    'default': 'both',
-                    'type': 'dropdown',
-                    'values': [('usenet & torrents', 'both'), ('usenet', 'nzb'), ('torrent', 'torrent')],
-                },
-                {
-                    'name': 'create_subdir',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Create a sub directory when saving the .nzb (or .torrent).',
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/deluge.py

@@ -1,20 +1,24 @@
 from base64 import b64encode, b16encode, b32decode
-from bencode import bencode as benc, bdecode
-from couchpotato.core.downloaders.base import Downloader, ReleaseDownloadList
-from couchpotato.core.helpers.encoding import isInt, sp
-from couchpotato.core.helpers.variable import tryFloat, cleanHost
-from couchpotato.core.logger import CPLog
 from datetime import timedelta
 from hashlib import sha1
-from synchronousdeluge import DelugeClient
 import os.path
 import re
 import traceback
 
+from bencode import bencode as benc, bdecode
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+from couchpotato.core.helpers.encoding import isInt, sp
+from couchpotato.core.helpers.variable import tryFloat, cleanHost
+from couchpotato.core.logger import CPLog
+from synchronousdeluge import DelugeClient
+
+
 log = CPLog(__name__)
 
+autoload = 'Deluge'
 
-class Deluge(Downloader):
+
+class Deluge(DownloaderBase):
 
     protocol = ['torrent', 'torrent_magnet']
     log = CPLog(__name__)
@@ -143,7 +147,7 @@ class Deluge(Downloader):
                 'seed_ratio': torrent['ratio'],
                 'timeleft': str(timedelta(seconds = torrent['eta'])),
                 'folder': sp(download_dir if len(torrent_files) == 1 else os.path.join(download_dir, torrent['name'])),
-                'files': '|'.join(torrent_files),
+                'files': torrent_files,
             })
 
         return release_downloads
@@ -291,3 +295,90 @@ class DelugeRPC(object):
             return torrent_hash
 
         return False
+
+
+config = [{
+    'name': 'deluge',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'deluge',
+            'label': 'Deluge',
+            'description': 'Use <a href="http://www.deluge-torrent.org/" target="_blank">Deluge</a> to download torrents.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'torrent',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:58846',
+                    'description': 'Hostname with port. Usually <strong>localhost:58846</strong>',
+                },
+                {
+                    'name': 'username',
+                },
+                {
+                    'name': 'password',
+                    'type': 'password',
+                },
+                {
+                    'name': 'directory',
+                    'type': 'directory',
+                    'description': 'Download to this directory. Keep empty for default Deluge download directory.',
+                },
+                {
+                    'name': 'completed_directory',
+                    'type': 'directory',
+                    'description': 'Move completed torrent to this directory. Keep empty for default Deluge options.',
+                    'advanced': True,
+                },
+                {
+                    'name': 'label',
+                    'description': 'Label to add to torrents in the Deluge UI.',
+                },
+                {
+                    'name': 'remove_complete',
+                    'label': 'Remove torrent',
+                    'type': 'bool',
+                    'default': True,
+                    'advanced': True,
+                    'description': 'Remove the torrent from Deluge after it has finished seeding.',
+                },
+                {
+                    'name': 'delete_files',
+                    'label': 'Remove files',
+                    'default': True,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Also remove the leftover files.',
+                },
+                {
+                    'name': 'paused',
+                    'type': 'bool',
+                    'advanced': True,
+                    'default': False,
+                    'description': 'Add the torrent paused.',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+                {
+                    'name': 'delete_failed',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Delete a release after the download has failed.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/deluge/__init__.py

@@ -1,91 +0,0 @@
-from .main import Deluge
-
-
-def start():
-    return Deluge()
-
-config = [{
-    'name': 'deluge',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'deluge',
-            'label': 'Deluge',
-            'description': 'Use <a href="http://www.deluge-torrent.org/" target="_blank">Deluge</a> to download torrents.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'torrent',
-                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:58846',
-                    'description': 'Hostname with port. Usually <strong>localhost:58846</strong>',
-                },
-                {
-                    'name': 'username',
-                },
-                {
-                    'name': 'password',
-                    'type': 'password',
-                },
-                {
-                    'name': 'directory',
-                    'type': 'directory',
-                    'description': 'Download to this directory. Keep empty for default Deluge download directory.',
-                },
-                {
-                    'name': 'completed_directory',
-                    'type': 'directory',
-                    'description': 'Move completed torrent to this directory. Keep empty for default Deluge options.',
-                    'advanced': True,
-                },
-                {
-                    'name': 'label',
-                    'description': 'Label to add to torrents in the Deluge UI.',
-                },
-                {
-                    'name': 'remove_complete',
-                    'label': 'Remove torrent',
-                    'type': 'bool',
-                    'default': True,
-                    'advanced': True,
-                    'description': 'Remove the torrent from Deluge after it has finished seeding.',
-                },
-                {
-                    'name': 'delete_files',
-                    'label': 'Remove files',
-                    'default': True,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Also remove the leftover files.',
-                },
-                {
-                    'name': 'paused',
-                    'type': 'bool',
-                    'advanced': True,
-                    'default': False,
-                    'description': 'Add the torrent paused.',
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-                {
-                    'name': 'delete_failed',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Delete a release after the download has failed.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/nzbget.py

@@ -1,8 +1,4 @@
 from base64 import standard_b64encode
-from couchpotato.core.downloaders.base import Downloader, ReleaseDownloadList
-from couchpotato.core.helpers.encoding import ss, sp
-from couchpotato.core.helpers.variable import tryInt, md5, cleanHost
-from couchpotato.core.logger import CPLog
 from datetime import timedelta
 import re
 import shutil
@@ -10,10 +6,18 @@ import socket
 import traceback
 import xmlrpclib
 
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+from couchpotato.core.helpers.encoding import ss, sp
+from couchpotato.core.helpers.variable import tryInt, md5, cleanHost
+from couchpotato.core.logger import CPLog
+
+
 log = CPLog(__name__)
 
+autoload = 'NZBGet'
 
-class NZBGet(Downloader):
+
+class NZBGet(DownloaderBase):
 
     protocol = ['nzb']
     rpc = 'xmlrpc'
@@ -142,7 +146,7 @@ class NZBGet(Downloader):
                     'timeleft': timeleft,
                 })
 
-        for nzb in queue: # 'Parameters' is not passed in rpc.postqueue
+        for nzb in queue:  # 'Parameters' is not passed in rpc.postqueue
             if nzb['NZBID'] in ids:
                 log.debug('Found %s in NZBGet postprocessing queue', nzb['NZBFilename'])
                 release_downloads.append({
@@ -214,3 +218,76 @@ class NZBGet(Downloader):
     def getRPC(self):
         url = cleanHost(host = self.conf('host'), ssl = self.conf('ssl'), username = self.conf('username'), password = self.conf('password')) + self.rpc
         return xmlrpclib.ServerProxy(url)
+
+
+config = [{
+    'name': 'nzbget',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'nzbget',
+            'label': 'NZBGet',
+            'description': 'Use <a href="http://nzbget.sourceforge.net/Main_Page" target="_blank">NZBGet</a> to download NZBs.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'nzb',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:6789',
+                    'description': 'Hostname with port. Usually <strong>localhost:6789</strong>',
+                },
+                {
+                    'name': 'ssl',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
+                },
+                {
+                    'name': 'username',
+                    'default': 'nzbget',
+                    'advanced': True,
+                    'description': 'Set a different username to connect. Default: nzbget',
+                },
+                {
+                    'name': 'password',
+                    'type': 'password',
+                    'description': 'Default NZBGet password is <i>tegbzn6789</i>',
+                },
+                {
+                    'name': 'category',
+                    'default': 'Movies',
+                    'description': 'The category CP places the nzb in. Like <strong>movies</strong> or <strong>couchpotato</strong>',
+                },
+                {
+                    'name': 'priority',
+                    'advanced': True,
+                    'default': '0',
+                    'type': 'dropdown',
+                    'values': [('Very Low', -100), ('Low', -50), ('Normal', 0), ('High', 50), ('Very High', 100)],
+                    'description': 'Only change this if you are using NZBget 9.0 or higher',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+                {
+                    'name': 'delete_failed',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Delete a release after the download has failed.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/nzbget/__init__.py

@@ -1,77 +0,0 @@
-from .main import NZBGet
-
-
-def start():
-    return NZBGet()
-
-config = [{
-    'name': 'nzbget',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'nzbget',
-            'label': 'NZBGet',
-            'description': 'Use <a href="http://nzbget.sourceforge.net/Main_Page" target="_blank">NZBGet</a> to download NZBs.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'nzb',
-                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:6789',
-                    'description': 'Hostname with port. Usually <strong>localhost:6789</strong>',
-                },
-                {
-                    'name': 'ssl',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
-                },
-                {
-                    'name': 'username',
-                    'default': 'nzbget',
-                    'advanced': True,
-                    'description': 'Set a different username to connect. Default: nzbget',
-                },
-                {
-                    'name': 'password',
-                    'type': 'password',
-                    'description': 'Default NZBGet password is <i>tegbzn6789</i>',
-                },
-                {
-                    'name': 'category',
-                    'default': 'Movies',
-                    'description': 'The category CP places the nzb in. Like <strong>movies</strong> or <strong>couchpotato</strong>',
-                },
-                {
-                    'name': 'priority',
-                    'advanced': True,
-                    'default': '0',
-                    'type': 'dropdown',
-                    'values': [('Very Low', -100), ('Low', -50), ('Normal', 0), ('High', 50), ('Very High', 100)],
-                    'description': 'Only change this if you are using NZBget 9.0 or higher',
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-                {
-                    'name': 'delete_failed',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Delete a release after the download has failed.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/nzbvortex.py

@@ -1,8 +1,4 @@
 from base64 import b64encode
-from couchpotato.core.downloaders.base import Downloader, ReleaseDownloadList
-from couchpotato.core.helpers.encoding import tryUrlencode, sp
-from couchpotato.core.helpers.variable import cleanHost
-from couchpotato.core.logger import CPLog
 from urllib2 import URLError
 from uuid import uuid4
 import hashlib
@@ -16,10 +12,18 @@ import time
 import traceback
 import urllib2
 
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+from couchpotato.core.helpers.encoding import tryUrlencode, sp
+from couchpotato.core.helpers.variable import cleanHost
+from couchpotato.core.logger import CPLog
+
+
 log = CPLog(__name__)
 
+autoload = 'NZBVortex'
 
-class NZBVortex(Downloader):
+
+class NZBVortex(DownloaderBase):
 
     protocol = ['nzb']
     api_level = None
@@ -186,3 +190,56 @@ class HTTPSConnection(httplib.HTTPSConnection):
 class HTTPSHandler(urllib2.HTTPSHandler):
     def https_open(self, req):
         return self.do_open(HTTPSConnection, req)
+
+
+config = [{
+    'name': 'nzbvortex',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'nzbvortex',
+            'label': 'NZBVortex',
+            'description': 'Use <a href="http://www.nzbvortex.com/landing/" target="_blank">NZBVortex</a> to download NZBs.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'nzb',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:4321',
+                    'description': 'Hostname with port. Usually <strong>localhost:4321</strong>',
+                },
+                {
+                    'name': 'ssl',
+                    'default': 1,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
+                },
+                {
+                    'name': 'api_key',
+                    'label': 'Api Key',
+                },
+                {
+                    'name': 'manual',
+                    'default': False,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+                {
+                    'name': 'delete_failed',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Delete a release after the download has failed.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/nzbvortex/__init__.py

@@ -1,57 +0,0 @@
-from .main import NZBVortex
-
-
-def start():
-    return NZBVortex()
-
-config = [{
-    'name': 'nzbvortex',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'nzbvortex',
-            'label': 'NZBVortex',
-            'description': 'Use <a href="http://www.nzbvortex.com/landing/" target="_blank">NZBVortex</a> to download NZBs.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'nzb',
-                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:4321',
-                    'description': 'Hostname with port. Usually <strong>localhost:4321</strong>',
-                },
-                {
-                    'name': 'ssl',
-                    'default': 1,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
-                },
-                {
-                    'name': 'api_key',
-                    'label': 'Api Key',
-                },
-                {
-                    'name': 'manual',
-                    'default': False,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-                {
-                    'name': 'delete_failed',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Delete a release after the download has failed.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/pneumatic.py

@@ -1,14 +1,18 @@
 from __future__ import with_statement
-from couchpotato.core.downloaders.base import Downloader
-from couchpotato.core.helpers.encoding import sp
-from couchpotato.core.logger import CPLog
 import os
 import traceback
 
+from couchpotato.core._base.downloader.main import DownloaderBase
+from couchpotato.core.helpers.encoding import sp
+from couchpotato.core.logger import CPLog
+
+
 log = CPLog(__name__)
 
+autoload = 'Pneumatic'
 
-class Pneumatic(Downloader):
+
+class Pneumatic(DownloaderBase):
 
     protocol = ['nzb']
     strm_syntax = 'plugin://plugin.program.pneumatic/?mode=strm&type=add_file&nzb=%s&nzbname=%s'
@@ -71,3 +75,37 @@ class Pneumatic(Downloader):
                 return True
 
         return False
+
+
+config = [{
+    'name': 'pneumatic',
+    'order': 30,
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'pneumatic',
+            'label': 'Pneumatic',
+            'description': 'Use <a href="http://forum.xbmc.org/showthread.php?tid=97657" target="_blank">Pneumatic</a> to download .strm files.',
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                },
+                {
+                    'name': 'directory',
+                    'type': 'directory',
+                    'description': 'Directory where the .strm file is saved to.',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/pneumatic/__init__.py

@@ -1,38 +0,0 @@
-from .main import Pneumatic
-
-
-def start():
-    return Pneumatic()
-
-config = [{
-    'name': 'pneumatic',
-    'order': 30,
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'pneumatic',
-            'label': 'Pneumatic',
-            'description': 'Use <a href="http://forum.xbmc.org/showthread.php?tid=97657" target="_blank">Pneumatic</a> to download .strm files.',
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                },
-                {
-                    'name': 'directory',
-                    'type': 'directory',
-                    'description': 'Directory where the .strm file is saved to.',
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/qbittorrent_.py

@@ -0,0 +1,245 @@
+from base64 import b16encode, b32decode
+from hashlib import sha1
+import os
+
+from bencode import bencode, bdecode
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+from couchpotato.core.helpers.encoding import sp
+from couchpotato.core.helpers.variable import cleanHost
+from couchpotato.core.logger import CPLog
+from qbittorrent.client import QBittorrentClient
+
+
+log = CPLog(__name__)
+
+autoload = 'qBittorrent'
+
+
+class qBittorrent(DownloaderBase):
+
+    protocol = ['torrent', 'torrent_magnet']
+    qb = None
+
+    def __init__(self):
+        super(qBittorrent, self).__init__()
+
+    def connect(self):
+        if self.qb is not None:
+            return self.qb
+
+        url = cleanHost(self.conf('host'), protocol = True, ssl = False)
+
+        if self.conf('username') and self.conf('password'):
+            self.qb = QBittorrentClient(
+                url,
+                username = self.conf('username'),
+                password = self.conf('password')
+            )
+        else:
+            self.qb = QBittorrentClient(url)
+
+        return self.qb
+
+    def test(self):
+        if self.connect():
+            return True
+
+        return False
+
+    def download(self, data = None, media = None, filedata = None):
+        if not media: media = {}
+        if not data: data = {}
+
+        log.debug('Sending "%s" to qBittorrent.', (data.get('name')))
+
+        if not self.connect():
+            return False
+
+        if not filedata and data.get('protocol') == 'torrent':
+            log.error('Failed sending torrent, no data')
+            return False
+
+
+        if data.get('protocol') == 'torrent_magnet':
+            filedata = self.magnetToTorrent(data.get('url'))
+
+            if filedata is False:
+                return False
+
+            data['protocol'] = 'torrent'
+
+        info = bdecode(filedata)["info"]
+        torrent_hash = sha1(bencode(info)).hexdigest()
+
+        # Convert base 32 to hex
+        if len(torrent_hash) == 32:
+            torrent_hash = b16encode(b32decode(torrent_hash))
+
+        # Send request to qBittorrent
+        try:
+            self.qb.add_file(filedata)
+
+            return self.downloadReturnId(torrent_hash)
+        except Exception as e:
+            log.error('Failed to send torrent to qBittorrent: %s', e)
+            return False
+
+    def getTorrentStatus(self, torrent):
+
+        if torrent.state in ('uploading', 'queuedUP', 'stalledUP'):
+            return 'seeding'
+
+        if torrent.progress == 1:
+            return 'completed'
+
+        return 'busy'
+
+    def getAllDownloadStatus(self, ids):
+        log.debug('Checking qBittorrent download status.')
+
+        if not self.connect():
+            return []
+
+        try:
+            torrents = self.qb.get_torrents()
+
+            release_downloads = ReleaseDownloadList(self)
+
+            for torrent in torrents:
+                if torrent.hash in ids:
+                    torrent.update_general() # get extra info
+                    torrent_filelist = torrent.get_files()
+
+                    torrent_files = []
+                    torrent_dir = os.path.join(torrent.save_path, torrent.name)
+
+                    if os.path.isdir(torrent_dir):
+                        torrent.save_path = torrent_dir
+
+                    if len(torrent_filelist) > 1 and os.path.isdir(torrent_dir): # multi file torrent, path.isdir check makes sure we're not in the root download folder
+                        for root, _, files in os.walk(torrent.save_path):
+                            for f in files:
+                                torrent_files.append(sp(os.path.join(root, f)))
+
+                    else: # multi or single file placed directly in torrent.save_path
+                        for f in torrent_filelist:
+                            file_path = os.path.join(torrent.save_path, f.name)
+                            if os.path.isfile(file_path):
+                                torrent_files.append(sp(file_path))
+
+                    release_downloads.append({
+                        'id': torrent.hash,
+                        'name': torrent.name,
+                        'status': self.getTorrentStatus(torrent),
+                        'seed_ratio': torrent.ratio,
+                        'original_status': torrent.state,
+                        'timeleft': torrent.progress * 100 if torrent.progress else -1, # percentage
+                        'folder': sp(torrent.save_path),
+                        'files': torrent_files
+                    })
+
+            return release_downloads
+
+        except Exception as e:
+            log.error('Failed to get status from qBittorrent: %s', e)
+            return []
+
+    def pause(self, release_download, pause = True):
+        if not self.connect():
+            return False
+
+        torrent = self.qb.get_torrent(release_download['id'])
+        if torrent is None:
+            return False
+
+        if pause:
+            return torrent.pause()
+        return torrent.resume()
+
+    def removeFailed(self, release_download):
+        log.info('%s failed downloading, deleting...', release_download['name'])
+        return self.processComplete(release_download, delete_files = True)
+
+    def processComplete(self, release_download, delete_files):
+        log.debug('Requesting qBittorrent to remove the torrent %s%s.',
+                  (release_download['name'], ' and cleanup the downloaded files' if delete_files else ''))
+
+        if not self.connect():
+            return False
+
+        torrent = self.qb.find_torrent(release_download['id'])
+
+        if torrent is None:
+            return False
+
+        if delete_files:
+            torrent.delete() # deletes torrent with data
+        else:
+            torrent.remove() # just removes the torrent, doesn't delete data
+
+        return True
+
+
+config = [{
+    'name': 'qbittorrent',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'qbittorrent',
+            'label': 'qbittorrent',
+            'description': '',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'torrent',
+                },
+                {
+                    'name': 'host',
+                    'default': 'http://localhost:8080/',
+                    'description': 'RPC Communication URI. Usually <strong>http://localhost:8080/</strong>'
+                },
+                {
+                    'name': 'username',
+                },
+                {
+                    'name': 'password',
+                    'type': 'password',
+                },
+                {
+                    'name': 'remove_complete',
+                    'label': 'Remove torrent',
+                    'default': False,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Remove the torrent after it finishes seeding.',
+                },
+                {
+                    'name': 'delete_files',
+                    'label': 'Remove files',
+                    'default': True,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Also remove the leftover files.',
+                },
+                {
+                    'name': 'paused',
+                    'type': 'bool',
+                    'advanced': True,
+                    'default': False,
+                    'description': 'Add the torrent paused.',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/rtorrent/__init__.py

@@ -1,100 +0,0 @@
-from .main import rTorrent
-
-
-def start():
-    return rTorrent()
-
-config = [{
-    'name': 'rtorrent',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'rtorrent',
-            'label': 'rTorrent',
-            'description': '',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'torrent',
-                },
-# @RuudBurger: How do I migrate this?
-#                {
-#                    'name': 'url',
-#                    'default': 'http://localhost:80/RPC2',
-#                    'description': 'XML-RPC Endpoint URI. Usually <strong>scgi://localhost:5000</strong> '
-#                                   'or <strong>http://localhost:80/RPC2</strong>'
-#                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:80',
-                    'description': 'RPC Communication URI. Usually <strong>scgi://localhost:5000</strong>, '
-                                   '<strong>httprpc://localhost/rutorrent</strong> or <strong>localhost:80</strong>'
-                },
-                {
-                    'name': 'ssl',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
-                },
-                {
-                    'name': 'rpc_url',
-                    'type': 'string',
-                    'default': 'RPC2',
-                    'advanced': True,
-                    'description': 'Change if your RPC mount is at a different path.',
-                },
-                {
-                    'name': 'username',
-                },
-                {
-                    'name': 'password',
-                    'type': 'password',
-                },
-                {
-                    'name': 'label',
-                    'description': 'Label to apply on added torrents.',
-                },
-                {
-                    'name': 'directory',
-                    'type': 'directory',
-                    'description': 'Download to this directory. Keep empty for default rTorrent download directory.',
-                },
-                {
-                    'name': 'remove_complete',
-                    'label': 'Remove torrent',
-                    'default': False,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Remove the torrent after it finishes seeding.',
-                },
-                {
-                    'name': 'delete_files',
-                    'label': 'Remove files',
-                    'default': True,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Also remove the leftover files.',
-                },
-                {
-                    'name': 'paused',
-                    'type': 'bool',
-                    'advanced': True,
-                    'default': False,
-                    'description': 'Add the torrent paused.',
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/rtorrent_.py

@@ -1,21 +1,26 @@
-from couchpotato.core.downloaders.base import Downloader, ReleaseDownloadList
-from couchpotato.core.event import fireEvent, addEvent
-from couchpotato.core.helpers.encoding import sp
-from couchpotato.core.helpers.variable import cleanHost, splitString
-from couchpotato.core.logger import CPLog
 from base64 import b16encode, b32decode
-from bencode import bencode, bdecode
 from datetime import timedelta
 from hashlib import sha1
-from rtorrent import RTorrent
-from rtorrent.err import MethodError
 from urlparse import urlparse
 import os
 
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+
+from couchpotato.core.event import addEvent
+from couchpotato.core.helpers.encoding import sp
+from couchpotato.core.helpers.variable import cleanHost, splitString
+from couchpotato.core.logger import CPLog
+from bencode import bencode, bdecode
+from rtorrent import RTorrent
+from scandir import scandir
+
+
 log = CPLog(__name__)
 
+autoload = 'rTorrent'
 
-class rTorrent(Downloader):
+
+class rTorrent(DownloaderBase):
 
     protocol = ['torrent', 'torrent_magnet']
     rt = None
@@ -54,27 +59,29 @@ class rTorrent(Downloader):
             return self.rt
 
         url = cleanHost(self.conf('host'), protocol = True, ssl = self.conf('ssl'))
+
+        # Automatically add '+https' to 'httprpc' protocol if SSL is enabled
+        if self.conf('ssl') and url.startswith('httprpc://'):
+            url = url.replace('httprpc://', 'httprpc+https://')
+
         parsed = urlparse(url)
 
         # rpc_url is only used on http/https scgi pass-through
         if parsed.scheme in ['http', 'https']:
             url += self.conf('rpc_url')
 
-        if self.conf('username') and self.conf('password'):
-            self.rt = RTorrent(
-                url,
-                self.conf('username'),
-                self.conf('password')
-            )
-        else:
-            self.rt = RTorrent(url)
+        self.rt = RTorrent(
+            url,
+            self.conf('username'),
+            self.conf('password')
+        )
 
         self.error_msg = ''
         try:
-             self.rt._verify_conn()
+            self.rt._verify_conn()
         except AssertionError as e:
-             self.error_msg = e.message
-             self.rt = None
+            self.error_msg = e.message
+            self.rt = None
 
         return self.rt
 
@@ -87,44 +94,6 @@ class rTorrent(Downloader):
 
         return False
 
-    def updateProviderGroup(self, name, data):
-        if data.get('seed_time'):
-            log.info('seeding time ignored, not supported')
-
-        if not name:
-            return False
-
-        if not self.connect():
-            return False
-
-        views = self.rt.get_views()
-
-        if name not in views:
-            self.rt.create_group(name)
-
-        group = self.rt.get_group(name)
-
-        try:
-            if data.get('seed_ratio'):
-                ratio = int(float(data.get('seed_ratio')) * 100)
-                log.debug('Updating provider ratio to %s, group name: %s', (ratio, name))
-
-                # Explicitly set all group options to ensure it is setup correctly
-                group.set_upload('1M')
-                group.set_min(ratio)
-                group.set_max(ratio)
-                group.set_command('d.stop')
-                group.enable()
-            else:
-                # Reset group action and disable it
-                group.set_command()
-                group.disable()
-        except MethodError as err:
-            log.error('Unable to set group options: %s', err.msg)
-            return False
-
-        return True
-
 
     def download(self, data = None, media = None, filedata = None):
         if not media: media = {}
@@ -135,10 +104,6 @@ class rTorrent(Downloader):
         if not self.connect():
             return False
 
-        group_name = 'cp_' + data.get('provider').lower()
-        if not self.updateProviderGroup(group_name, data):
-            return False
-
         torrent_params = {}
         if self.conf('label'):
             torrent_params['label'] = self.conf('label')
@@ -179,9 +144,6 @@ class rTorrent(Downloader):
             if self.conf('directory'):
                 torrent.set_directory(self.conf('directory'))
 
-            # Set Ratio Group
-            torrent.set_visible(group_name)
-
             # Start torrent
             if not self.conf('paused', default = 0):
                 torrent.start()
@@ -238,7 +200,7 @@ class rTorrent(Downloader):
                         'original_status': torrent.state,
                         'timeleft': str(timedelta(seconds = float(torrent.left_bytes) / torrent.down_rate)) if torrent.down_rate > 0 else -1,
                         'folder': sp(torrent.directory),
-                        'files': '|'.join(torrent_files)
+                        'files': torrent_files
                     })
 
             return release_downloads
@@ -282,7 +244,7 @@ class rTorrent(Downloader):
             if torrent.is_multi_file() and torrent.directory.endswith(torrent.name):
                 # Remove empty directories bottom up
                 try:
-                    for path, _, _ in os.walk(torrent.directory, topdown = False):
+                    for path, _, _ in scandir.walk(torrent.directory, topdown = False):
                         os.rmdir(path)
                 except OSError:
                     log.info('Directory "%s" contains extra files, unable to remove', torrent.directory)
@@ -290,3 +252,92 @@ class rTorrent(Downloader):
         torrent.erase() # just removes the torrent, doesn't delete data
 
         return True
+
+
+config = [{
+    'name': 'rtorrent',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'rtorrent',
+            'label': 'rTorrent',
+            'description': '',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'torrent',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:80',
+                    'description': 'RPC Communication URI. Usually <strong>scgi://localhost:5000</strong>, '
+                                   '<strong>httprpc://localhost/rutorrent</strong> or <strong>localhost:80</strong>'
+                },
+                {
+                    'name': 'ssl',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
+                },
+                {
+                    'name': 'rpc_url',
+                    'type': 'string',
+                    'default': 'RPC2',
+                    'advanced': True,
+                    'description': 'Change if your RPC mount is at a different path.',
+                },
+                {
+                    'name': 'username',
+                },
+                {
+                    'name': 'password',
+                    'type': 'password',
+                },
+                {
+                    'name': 'label',
+                    'description': 'Label to apply on added torrents.',
+                },
+                {
+                    'name': 'directory',
+                    'type': 'directory',
+                    'description': 'Download to this directory. Keep empty for default rTorrent download directory.',
+                },
+                {
+                    'name': 'remove_complete',
+                    'label': 'Remove torrent',
+                    'default': False,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Remove the torrent after it finishes seeding.',
+                },
+                {
+                    'name': 'delete_files',
+                    'label': 'Remove files',
+                    'default': True,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Also remove the leftover files.',
+                },
+                {
+                    'name': 'paused',
+                    'type': 'bool',
+                    'advanced': True,
+                    'default': False,
+                    'description': 'Add the torrent paused.',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/sabnzbd.py

@@ -1,18 +1,22 @@
-from couchpotato.core.downloaders.base import Downloader, ReleaseDownloadList
-from couchpotato.core.helpers.encoding import tryUrlencode, ss, sp
-from couchpotato.core.helpers.variable import cleanHost, mergeDicts
-from couchpotato.core.logger import CPLog
-from couchpotato.environment import Env
 from datetime import timedelta
 from urllib2 import URLError
 import json
 import os
 import traceback
 
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+from couchpotato.core.helpers.encoding import tryUrlencode, ss, sp
+from couchpotato.core.helpers.variable import cleanHost, mergeDicts
+from couchpotato.core.logger import CPLog
+from couchpotato.environment import Env
+
+
 log = CPLog(__name__)
 
+autoload = 'Sabnzbd'
 
-class Sabnzbd(Downloader):
+
+class Sabnzbd(DownloaderBase):
 
     protocol = ['nzb']
 
@@ -201,3 +205,77 @@ class Sabnzbd(Downloader):
         else:
             return data
 
+
+config = [{
+    'name': 'sabnzbd',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'sabnzbd',
+            'label': 'Sabnzbd',
+            'description': 'Use <a href="http://sabnzbd.org/" target="_blank">SABnzbd</a> (0.7+) to download NZBs.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'nzb',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:8080',
+                },
+                {
+                    'name': 'ssl',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
+                },
+                {
+                    'name': 'api_key',
+                    'label': 'Api Key',
+                    'description': 'Used for all calls to Sabnzbd.',
+                },
+                {
+                    'name': 'category',
+                    'label': 'Category',
+                    'description': 'The category CP places the nzb in. Like <strong>movies</strong> or <strong>couchpotato</strong>',
+                },
+                {
+                    'name': 'priority',
+                    'label': 'Priority',
+                    'type': 'dropdown',
+                    'default': '0',
+                    'advanced': True,
+                    'values': [('Paused', -2), ('Low', -1), ('Normal', 0), ('High', 1), ('Forced', 2)],
+                    'description': 'Add to the queue with this priority.',
+                },
+                {
+                    'name': 'manual',
+                    'default': False,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+                {
+                    'name': 'remove_complete',
+                    'advanced': True,
+                    'label': 'Remove NZB',
+                    'default': False,
+                    'type': 'bool',
+                    'description': 'Remove the NZB from history after it completed.',
+                },
+                {
+                    'name': 'delete_failed',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Delete a release after the download has failed.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/sabnzbd/__init__.py

@@ -1,79 +0,0 @@
-from .main import Sabnzbd
-
-
-def start():
-    return Sabnzbd()
-
-config = [{
-    'name': 'sabnzbd',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'sabnzbd',
-            'label': 'Sabnzbd',
-            'description': 'Use <a href="http://sabnzbd.org/" target="_blank">SABnzbd</a> (0.7+) to download NZBs.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'nzb',
-                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:8080',
-                },
-                {
-                    'name': 'ssl',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Use HyperText Transfer Protocol Secure, or <strong>https</strong>',
-                },
-                {
-                    'name': 'api_key',
-                    'label': 'Api Key',
-                    'description': 'Used for all calls to Sabnzbd.',
-                },
-                {
-                    'name': 'category',
-                    'label': 'Category',
-                    'description': 'The category CP places the nzb in. Like <strong>movies</strong> or <strong>couchpotato</strong>',
-                },
-                {
-                    'name': 'priority',
-                    'label': 'Priority',
-                    'type': 'dropdown',
-                    'default': '0',
-                    'advanced': True,
-                    'values': [('Paused', -2), ('Low', -1), ('Normal', 0), ('High', 1), ('Forced', 2)],
-                    'description': 'Add to the queue with this priority.',
-                },
-                {
-                    'name': 'manual',
-                    'default': False,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-                {
-                    'name': 'remove_complete',
-                    'advanced': True,
-                    'label': 'Remove NZB',
-                    'default': False,
-                    'type': 'bool',
-                    'description': 'Remove the NZB from history after it completed.',
-                },
-                {
-                    'name': 'delete_failed',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Delete a release after the download has failed.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/synology.py

@@ -1,15 +1,19 @@
-from couchpotato.core.downloaders.base import Downloader
+import json
+import traceback
+
+from couchpotato.core._base.downloader.main import DownloaderBase
 from couchpotato.core.helpers.encoding import isInt
 from couchpotato.core.helpers.variable import cleanHost
 from couchpotato.core.logger import CPLog
-import json
 import requests
-import traceback
+
 
 log = CPLog(__name__)
 
+autoload = 'Synology'
 
-class Synology(Downloader):
+
+class Synology(DownloaderBase):
 
     protocol = ['nzb', 'torrent', 'torrent_magnet']
     status_support = False
@@ -29,7 +33,7 @@ class Synology(Downloader):
 
         try:
             # Send request to Synology
-            srpc = SynologyRPC(host[0], host[1], self.conf('username'), self.conf('password'))
+            srpc = SynologyRPC(host[0], host[1], self.conf('username'), self.conf('password'), self.conf('destination'))
             if data['protocol'] == 'torrent_magnet':
                 log.info('Adding torrent URL %s', data['url'])
                 response = srpc.create_task(url = data['url'])
@@ -80,7 +84,7 @@ class SynologyRPC(object):
 
     """SynologyRPC lite library"""
 
-    def __init__(self, host = 'localhost', port = 5000, username = None, password = None):
+    def __init__(self, host = 'localhost', port = 5000, username = None, password = None, destination = None):
 
         super(SynologyRPC, self).__init__()
 
@@ -88,6 +92,7 @@ class SynologyRPC(object):
         self.auth_url = 'http://%s:%s/webapi/auth.cgi' % (host, port)
         self.username = username
         self.password = password
+        self.destination = destination
         self.session_name = 'DownloadStation'
 
     def _login(self):
@@ -140,6 +145,10 @@ class SynologyRPC(object):
                     'version': '1',
                     'method': 'create',
                     '_sid': self.sid}
+
+            if self.destination and len(self.destination) > 0:
+                args['destination'] = self.destination
+
             if url:
                 log.info('Login success, adding torrent URI')
                 args['uri'] = url
@@ -160,3 +169,57 @@ class SynologyRPC(object):
 
     def test(self):
         return bool(self._login())
+
+
+config = [{
+    'name': 'synology',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'synology',
+            'label': 'Synology',
+            'description': 'Use <a href="http://www.synology.com/dsm/home_home_applications_download_station.php" target="_blank">Synology Download Station</a> to download.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'nzb,torrent',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:5000',
+                    'description': 'Hostname with port. Usually <strong>localhost:5000</strong>',
+                },
+                {
+                    'name': 'username',
+                },
+                {
+                    'name': 'password',
+                    'type': 'password',
+                },
+                {
+                    'name': 'destination',
+                    'description': 'Specify <strong>existing</strong> destination share to where your files will be downloaded, usually <strong>Downloads</strong>',
+                    'advanced': True,
+                },
+                {
+                    'name': 'use_for',
+                    'label': 'Use for',
+                    'default': 'both',
+                    'type': 'dropdown',
+                    'values': [('usenet & torrents', 'both'), ('usenet', 'nzb'), ('torrent', 'torrent')],
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/synology/__init__.py

@@ -1,53 +0,0 @@
-from .main import Synology
-
-
-def start():
-    return Synology()
-
-config = [{
-    'name': 'synology',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'synology',
-            'label': 'Synology',
-            'description': 'Use <a href="http://www.synology.com/dsm/home_home_applications_download_station.php" target="_blank">Synology Download Station</a> to download.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'nzb,torrent',
-                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:5000',
-                    'description': 'Hostname with port. Usually <strong>localhost:5000</strong>',
-                },
-                {
-                    'name': 'username',
-                },
-                {
-                    'name': 'password',
-                    'type': 'password',
-                },
-                {
-                    'name': 'use_for',
-                    'label': 'Use for',
-                    'default': 'both',
-                    'type': 'dropdown',
-                    'values': [('usenet & torrents', 'both'), ('usenet', 'nzb'), ('torrent', 'torrent')],
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/transmission.py

@@ -1,8 +1,4 @@
 from base64 import b64encode
-from couchpotato.core.downloaders.base import Downloader, ReleaseDownloadList
-from couchpotato.core.helpers.encoding import isInt, sp
-from couchpotato.core.helpers.variable import tryInt, tryFloat, cleanHost
-from couchpotato.core.logger import CPLog
 from datetime import timedelta
 import httplib
 import json
@@ -10,10 +6,18 @@ import os.path
 import re
 import urllib2
 
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+from couchpotato.core.helpers.encoding import isInt, sp
+from couchpotato.core.helpers.variable import tryInt, tryFloat, cleanHost
+from couchpotato.core.logger import CPLog
+
+
 log = CPLog(__name__)
 
+autoload = 'Transmission'
 
-class Transmission(Downloader):
+
+class Transmission(DownloaderBase):
 
     protocol = ['torrent', 'torrent_magnet']
     log = CPLog(__name__)
@@ -137,7 +141,7 @@ class Transmission(Downloader):
                     'seed_ratio': torrent['uploadRatio'],
                     'timeleft': str(timedelta(seconds = torrent['eta'])),
                     'folder': sp(torrent_folder if len(torrent_files) == 1 else os.path.join(torrent_folder, torrent['name'])),
-                    'files': '|'.join(torrent_files)
+                    'files': torrent_files
                 })
 
         return release_downloads
@@ -156,6 +160,7 @@ class Transmission(Downloader):
         log.debug('Requesting Transmission to remove the torrent %s%s.', (release_download['name'], ' and cleanup the downloaded files' if delete_files else ''))
         return self.trpc.remove_torrent(release_download['id'], delete_files)
 
+
 class TransmissionRPC(object):
 
     """TransmissionRPC lite library"""
@@ -251,3 +256,93 @@ class TransmissionRPC(object):
         post_data = {'arguments': {'ids': torrent_id, 'delete-local-data': delete_local_data}, 'method': 'torrent-remove', 'tag': self.tag}
         return self._request(post_data)
 
+
+config = [{
+    'name': 'transmission',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'transmission',
+            'label': 'Transmission',
+            'description': 'Use <a href="http://www.transmissionbt.com/" target="_blank">Transmission</a> to download torrents.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'torrent',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:9091',
+                    'description': 'Hostname with port. Usually <strong>localhost:9091</strong>',
+                },
+                {
+                    'name': 'rpc_url',
+                    'type': 'string',
+                    'default': 'transmission',
+                    'advanced': True,
+                    'description': 'Change if you don\'t run Transmission RPC at the default url.',
+                },
+                {
+                    'name': 'username',
+                },
+                {
+                    'name': 'password',
+                    'type': 'password',
+                },
+                {
+                    'name': 'directory',
+                    'type': 'directory',
+                    'description': 'Download to this directory. Keep empty for default Transmission download directory.',
+                },
+                {
+                    'name': 'remove_complete',
+                    'label': 'Remove torrent',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Remove the torrent from Transmission after it finished seeding.',
+                },
+                {
+                    'name': 'delete_files',
+                    'label': 'Remove files',
+                    'default': True,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Also remove the leftover files.',
+                },
+                {
+                    'name': 'paused',
+                    'type': 'bool',
+                    'advanced': True,
+                    'default': False,
+                    'description': 'Add the torrent paused.',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+                {
+                    'name': 'stalled_as_failed',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Consider a stalled torrent as failed',
+                },
+                {
+                    'name': 'delete_failed',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Delete a release after the download has failed.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/transmission/__init__.py

@@ -1,95 +0,0 @@
-from .main import Transmission
-
-
-def start():
-    return Transmission()
-
-config = [{
-    'name': 'transmission',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'transmission',
-            'label': 'Transmission',
-            'description': 'Use <a href="http://www.transmissionbt.com/" target="_blank">Transmission</a> to download torrents.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'torrent',
-                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:9091',
-                    'description': 'Hostname with port. Usually <strong>localhost:9091</strong>',
-                },
-                {
-                    'name': 'rpc_url',
-                    'type': 'string',
-                    'default': 'transmission',
-                    'advanced': True,
-                    'description': 'Change if you don\'t run Transmission RPC at the default url.',
-                },
-                {
-                    'name': 'username',
-                },
-                {
-                    'name': 'password',
-                    'type': 'password',
-                },
-                {
-                    'name': 'directory',
-                    'type': 'directory',
-                    'description': 'Download to this directory. Keep empty for default Transmission download directory.',
-                },
-                {
-                    'name': 'remove_complete',
-                    'label': 'Remove torrent',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Remove the torrent from Transmission after it finished seeding.',
-                },
-                {
-                    'name': 'delete_files',
-                    'label': 'Remove files',
-                    'default': True,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Also remove the leftover files.',
-                },
-                {
-                    'name': 'paused',
-                    'type': 'bool',
-                    'advanced': True,
-                    'default': False,
-                    'description': 'Add the torrent paused.',
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-                {
-                    'name': 'stalled_as_failed',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Consider a stalled torrent as failed',
-                },
-                {
-                    'name': 'delete_failed',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Delete a release after the download has failed.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/downloaders/utorrent.py

@@ -1,12 +1,6 @@
 from base64 import b16encode, b32decode
-from bencode import bencode as benc, bdecode
-from couchpotato.core.downloaders.base import Downloader, ReleaseDownloadList
-from couchpotato.core.helpers.encoding import isInt, ss, sp
-from couchpotato.core.helpers.variable import tryInt, tryFloat, cleanHost
-from couchpotato.core.logger import CPLog
 from datetime import timedelta
 from hashlib import sha1
-from multipartpost import MultipartPostHandler
 import cookielib
 import httplib
 import json
@@ -17,22 +11,32 @@ import time
 import urllib
 import urllib2
 
+from bencode import bencode as benc, bdecode
+from couchpotato.core._base.downloader.main import DownloaderBase, ReleaseDownloadList
+from couchpotato.core.helpers.encoding import isInt, ss, sp
+from couchpotato.core.helpers.variable import tryInt, tryFloat, cleanHost
+from couchpotato.core.logger import CPLog
+from multipartpost import MultipartPostHandler
+
+
 log = CPLog(__name__)
 
+autoload = 'uTorrent'
 
-class uTorrent(Downloader):
+
+class uTorrent(DownloaderBase):
 
     protocol = ['torrent', 'torrent_magnet']
     utorrent_api = None
     status_flags = {
-        'STARTED'     : 1,
-        'CHECKING'    : 2,
-        'CHECK-START' : 4,
-        'CHECKED'     : 8,
-        'ERROR'       : 16,
-        'PAUSED'      : 32,
-        'QUEUED'      : 64,
-        'LOADED'      : 128
+        'STARTED': 1,
+        'CHECKING': 2,
+        'CHECK-START': 4,
+        'CHECKED': 8,
+        'ERROR': 16,
+        'PAUSED': 32,
+        'QUEUED': 64,
+        'LOADED': 128
     }
 
     def connect(self):
@@ -180,7 +184,7 @@ class uTorrent(Downloader):
                     'original_status': torrent[1],
                     'timeleft': str(timedelta(seconds = torrent[10])),
                     'folder': sp(torrent[26]),
-                    'files': '|'.join(torrent_files)
+                    'files': torrent_files
                 })
 
         return release_downloads
@@ -340,3 +344,79 @@ class uTorrentAPI(object):
             return False
         response = json.loads(data)
         return int(response.get('build'))
+
+
+config = [{
+    'name': 'utorrent',
+    'groups': [
+        {
+            'tab': 'downloaders',
+            'list': 'download_providers',
+            'name': 'utorrent',
+            'label': 'uTorrent',
+            'description': 'Use <a href="http://www.utorrent.com/" target="_blank">uTorrent</a> (3.0+) to download torrents.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'default': 0,
+                    'type': 'enabler',
+                    'radio_group': 'torrent',
+                },
+                {
+                    'name': 'host',
+                    'default': 'localhost:8000',
+                    'description': 'Port can be found in settings when enabling WebUI.',
+                },
+                {
+                    'name': 'username',
+                },
+                {
+                    'name': 'password',
+                    'type': 'password',
+                },
+                {
+                    'name': 'label',
+                    'description': 'Label to add torrent as.',
+                },
+                {
+                    'name': 'remove_complete',
+                    'label': 'Remove torrent',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Remove the torrent from uTorrent after it finished seeding.',
+                },
+                {
+                    'name': 'delete_files',
+                    'label': 'Remove files',
+                    'default': True,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Also remove the leftover files.',
+                },
+                {
+                    'name': 'paused',
+                    'type': 'bool',
+                    'advanced': True,
+                    'default': False,
+                    'description': 'Add the torrent paused.',
+                },
+                {
+                    'name': 'manual',
+                    'default': 0,
+                    'type': 'bool',
+                    'advanced': True,
+                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
+                },
+                {
+                    'name': 'delete_failed',
+                    'default': True,
+                    'advanced': True,
+                    'type': 'bool',
+                    'description': 'Delete a release after the download has failed.',
+                },
+            ],
+        }
+    ],
+}]

couchpotato/core/downloaders/utorrent/__init__.py

@@ -1,80 +0,0 @@
-from .main import uTorrent
-
-
-def start():
-    return uTorrent()
-
-config = [{
-    'name': 'utorrent',
-    'groups': [
-        {
-            'tab': 'downloaders',
-            'list': 'download_providers',
-            'name': 'utorrent',
-            'label': 'uTorrent',
-            'description': 'Use <a href="http://www.utorrent.com/" target="_blank">uTorrent</a> (3.0+) to download torrents.',
-            'wizard': True,
-            'options': [
-                {
-                    'name': 'enabled',
-                    'default': 0,
-                    'type': 'enabler',
-                    'radio_group': 'torrent',
-                },
-                {
-                    'name': 'host',
-                    'default': 'localhost:8000',
-                    'description': 'Port can be found in settings when enabling WebUI.',
-                },
-                {
-                    'name': 'username',
-                },
-                {
-                    'name': 'password',
-                    'type': 'password',
-                },
-                {
-                    'name': 'label',
-                    'description': 'Label to add torrent as.',
-                },
-                {
-                    'name': 'remove_complete',
-                    'label': 'Remove torrent',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Remove the torrent from uTorrent after it finished seeding.',
-                },
-                {
-                    'name': 'delete_files',
-                    'label': 'Remove files',
-                    'default': True,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Also remove the leftover files.',
-                },
-                {
-                    'name': 'paused',
-                    'type': 'bool',
-                    'advanced': True,
-                    'default': False,
-                    'description': 'Add the torrent paused.',
-                },
-                {
-                    'name': 'manual',
-                    'default': 0,
-                    'type': 'bool',
-                    'advanced': True,
-                    'description': 'Disable this downloader for automated searches, but use it when I manually send a release.',
-                },
-                {
-                    'name': 'delete_failed',
-                    'default': True,
-                    'advanced': True,
-                    'type': 'bool',
-                    'description': 'Delete a release after the download has failed.',
-                },
-            ],
-        }
-    ],
-}]

couchpotato/core/event.py

@@ -1,8 +1,10 @@
+import threading
+import traceback
+
 from axl.axel import Event
 from couchpotato.core.helpers.variable import mergeDicts, natsortKey
 from couchpotato.core.logger import CPLog
-import threading
-import traceback
+
 
 log = CPLog(__name__)
 events = {}

couchpotato/core/helpers/encoding.py

@@ -1,12 +1,14 @@
-from couchpotato.core.logger import CPLog
 from string import ascii_letters, digits
 from urllib import quote_plus
 import os
 import re
 import traceback
 import unicodedata
+
+from couchpotato.core.logger import CPLog
 import six
 
+
 log = CPLog(__name__)
 
 
@@ -76,7 +78,7 @@ def sp(path, *args):
     # Replace *NIX ambiguous '//' at the beginning of a path with '/' (crashes guessit)
     path = re.sub('^//', '/', path)
 
-    return path
+    return toUnicode(path)
 
 
 def ek(original, *args):

couchpotato/core/helpers/request.py

@@ -1,8 +1,9 @@
-from couchpotato.core.helpers.encoding import toUnicode
-from couchpotato.core.helpers.variable import natsortKey
 from urllib import unquote
 import re
 
+from couchpotato.core.helpers.encoding import toUnicode
+from couchpotato.core.helpers.variable import natsortKey
+
 
 def getParams(params):
 
@@ -42,6 +43,7 @@ def getParams(params):
 
     return dictToList(temp)
 
+non_decimal = re.compile(r'[^\d.]+')
 
 def dictToList(params):
 
@@ -52,7 +54,15 @@ def dictToList(params):
                 convert = lambda text: int(text) if text.isdigit() else text.lower()
                 alphanum_key = lambda key: [convert(c) for c in re.split('([0-9]+)', key)]
                 sorted_keys = sorted(value.keys(), key = alphanum_key)
-                new_value = [dictToList(value[k]) for k in sorted_keys]
+
+                all_ints = 0
+                for pnr in sorted_keys:
+                    all_ints += 1 if non_decimal.sub('', pnr) == pnr else 0
+
+                if all_ints == len(sorted_keys):
+                    new_value = [dictToList(value[k]) for k in sorted_keys]
+                else:
+                    new_value = value
             except:
                 new_value = value
 

couchpotato/core/helpers/rss.py

@@ -1,6 +1,8 @@
-from couchpotato.core.logger import CPLog
 import xml.etree.ElementTree as XMLTree
 
+from couchpotato.core.logger import CPLog
+
+
 log = CPLog(__name__)
 
 

couchpotato/core/helpers/variable.py

@@ -1,5 +1,3 @@
-from couchpotato.core.helpers.encoding import simplifyString, toSafeString, ss
-from couchpotato.core.logger import CPLog
 import collections
 import hashlib
 import os
@@ -8,9 +6,13 @@ import random
 import re
 import string
 import sys
+
+from couchpotato.core.helpers.encoding import simplifyString, toSafeString, ss
+from couchpotato.core.logger import CPLog
 import six
 from six.moves import map, zip, filter
 
+
 log = CPLog(__name__)
 
 
@@ -214,6 +216,7 @@ def tryFloat(s):
             return float(s)
     except: return 0
 
+
 def natsortKey(string_):
     """See http://www.codinghorror.com/blog/archives/001018.html"""
     return [int(s) if s.isdigit() else s for s in re.split(r'(\d+)', string_)]
@@ -225,26 +228,28 @@ def toIterable(value):
     return [value]
 
 
-def getTitle(library_dict):
+def getIdentifier(media):
+    return media.get('identifier') or media.get('identifiers', {}).get('imdb')
+
+
+def getTitle(media_dict):
     try:
         try:
-            return library_dict['titles'][0]['title']
+            return media_dict['title']
         except:
             try:
-                for title in library_dict.titles:
-                    if title.default:
-                        return title.title
+                return media_dict['titles'][0]
             except:
                 try:
-                    return library_dict['info']['titles'][0]
+                    return media_dict['info']['titles'][0]
                 except:
-                    log.error('Could not get title for %s', library_dict.identifier)
-                    return None
-
-        log.error('Could not get title for %s', library_dict['identifier'])
-        return None
+                    try:
+                        return media_dict['media']['info']['titles'][0]
+                    except:
+                        log.error('Could not get title for %s', getIdentifier(media_dict))
+                        return None
     except:
-        log.error('Could not get title for library item: %s', library_dict)
+        log.error('Could not get title for library item: %s', media_dict)
         return None
 
 
@@ -289,8 +294,11 @@ def isSubFolder(sub_folder, base_folder):
     # Returns True if sub_folder is the same as or inside base_folder
     return base_folder and sub_folder and ss(os.path.normpath(base_folder).rstrip(os.path.sep) + os.path.sep) in ss(os.path.normpath(sub_folder).rstrip(os.path.sep) + os.path.sep)
 
+
 # From SABNZBD
-re_password = [re.compile(r'([^/\\]+)[/\\](.+)'), re.compile(r'(.+){{([^{}]+)}}$'), re.compile(r'(.+)\s+password\s*=\s*(.+)$', re.I)]
+re_password = [re.compile(r'(.+){{([^{}]+)}}$'), re.compile(r'(.+)\s+password\s*=\s*(.+)$', re.I)]
+
+
 def scanForPassword(name):
     m = None
     for reg in re_password:

couchpotato/core/loader.py

@@ -1,31 +1,33 @@
-from couchpotato.core.event import fireEvent
-from couchpotato.core.logger import CPLog
-from importhelper import import_module
 import os
 import sys
 import traceback
+
+from couchpotato.core.event import fireEvent
+from couchpotato.core.logger import CPLog
+from importhelper import import_module
 import six
 
+
 log = CPLog(__name__)
 
 
 class Loader(object):
-    plugins = {}
-    providers = {}
-    modules = {}
+
+    def __init__(self):
+        self.plugins = {}
+        self.providers = {}
+        self.modules = {}
+        self.paths = {}
 
     def preload(self, root = ''):
         core = os.path.join(root, 'couchpotato', 'core')
 
-        self.paths = {
+        self.paths.update({
             'core': (0, 'couchpotato.core._base', os.path.join(core, '_base')),
             'plugin': (1, 'couchpotato.core.plugins', os.path.join(core, 'plugins')),
             'notifications': (20, 'couchpotato.core.notifications', os.path.join(core, 'notifications')),
             'downloaders': (20, 'couchpotato.core.downloaders', os.path.join(core, 'downloaders')),
-        }
-
-        # Add providers to loader
-        self.addPath(root, ['couchpotato', 'core', 'providers'], 25, recursive = False)
+        })
 
         # Add media to loader
         self.addPath(root, ['couchpotato', 'core', 'media'], 25, recursive = True)
@@ -57,12 +59,10 @@ class Loader(object):
                     if m is None:
                         continue
 
-                    log.info('Loading %s: %s', (plugin['type'], plugin['name']))
-
                     # Save default settings for plugin/provider
                     did_save += self.loadSettings(m, module_name, save = False)
 
-                    self.loadPlugins(m, plugin.get('name'))
+                    self.loadPlugins(m, plugin.get('type'), plugin.get('name'))
                 except ImportError as e:
                     # todo:: subclass ImportError for missing requirements.
                     if e.message.lower().startswith("missing"):
@@ -96,14 +96,19 @@ class Loader(object):
             self.addModule(priority, plugin_type, module, os.path.basename(dir_name))
 
         for name in os.listdir(dir_name):
-            if os.path.isdir(os.path.join(dir_name, name)) and name != 'static' and os.path.isfile(os.path.join(dir_name, name, '__init__.py')):
+            path = os.path.join(dir_name, name)
+            ext = os.path.splitext(path)[1]
+            ext_length = len(ext)
+            if name != 'static' and ((os.path.isdir(path) and os.path.isfile(os.path.join(path, '__init__.py')))
+                                     or (os.path.isfile(path) and ext == '.py')):
+                name = name[:-ext_length] if ext_length > 0 else name
                 module_name = '%s.%s' % (module, name)
                 self.addModule(priority, plugin_type, module_name, name)
 
     def loadSettings(self, module, name, save = True):
 
         if not hasattr(module, 'config'):
-            log.debug('Skip loading settings for plugin %s as it has no config section' % module.__file__)
+            #log.debug('Skip loading settings for plugin %s as it has no config section' % module.__file__)
             return False
 
         try:
@@ -119,13 +124,20 @@ class Loader(object):
             log.debug('Failed loading settings for "%s": %s', (name, traceback.format_exc()))
             return False
 
-    def loadPlugins(self, module, name):
+    def loadPlugins(self, module, type, name):
 
-        if not hasattr(module, 'start'):
-            log.debug('Skip startup for plugin %s as it has no start section' % module.__file__)
+        if not hasattr(module, 'autoload'):
+            #log.debug('Skip startup for plugin %s as it has no start section' % module.__file__)
             return False
         try:
-            module.start()
+            # Load single file plugin
+            if isinstance(module.autoload, (str, unicode)):
+                getattr(module, module.autoload)()
+            # Load folder plugin
+            else:
+                module.autoload()
+
+            log.info('Loaded %s: %s', (type, name))
             return True
         except:
             log.error('Failed loading plugin "%s": %s', (module.__file__, traceback.format_exc()))
@@ -137,6 +149,9 @@ class Loader(object):
             self.modules[priority] = {}
 
         module = module.lstrip('.')
+        if plugin_type.startswith('couchpotato_core'):
+            plugin_type = plugin_type[17:]
+
         self.modules[priority][module] = {
             'priority': priority,
             'module': module,

couchpotato/core/logger.py

@@ -7,6 +7,9 @@ class CPLog(object):
     context = ''
     replace_private = ['api', 'apikey', 'api_key', 'password', 'username', 'h', 'uid', 'key', 'passkey']
 
+    Env = None
+    is_develop = False
+
     def __init__(self, context = ''):
         if context.endswith('.main'):
             context = context[:-5]
@@ -14,6 +17,14 @@ class CPLog(object):
         self.context = context
         self.logger = logging.getLogger()
 
+    def setup(self):
+
+        if not self.Env:
+            from couchpotato.environment import Env
+
+            self.Env = Env
+            self.is_develop = Env.get('dev')
+
     def info(self, msg, replace_tuple = ()):
         self.logger.info(self.addContext(msg, replace_tuple))
 
@@ -37,7 +48,6 @@ class CPLog(object):
 
     def safeMessage(self, msg, replace_tuple = ()):
 
-        from couchpotato.environment import Env
         from couchpotato.core.helpers.encoding import ss, toUnicode
 
         msg = ss(msg)
@@ -53,7 +63,8 @@ class CPLog(object):
             except Exception as e:
                 self.logger.error('Failed encoding stuff to log "%s": %s' % (msg, e))
 
-        if not Env.get('dev'):
+        self.setup()
+        if not self.is_develop:
 
             for replace in self.replace_private:
                 msg = re.sub('(\?%s=)[^\&]+' % replace, '?%s=xxx' % replace, msg)
@@ -61,7 +72,7 @@ class CPLog(object):
 
             # Replace api key
             try:
-                api_key = Env.setting('api_key')
+                api_key = self.Env.setting('api_key')
                 if api_key:
                     msg = msg.replace(api_key, 'API_KEY')
             except:

couchpotato/core/media/__init__.py

@@ -1,8 +1,12 @@
+import os
 import traceback
-from couchpotato import get_session, CPLog
-from couchpotato.core.event import addEvent, fireEventAsync, fireEvent
+
+from couchpotato import get_db, CPLog
+from couchpotato.core.event import addEvent, fireEvent, fireEventAsync
+from couchpotato.core.helpers.encoding import toUnicode
 from couchpotato.core.plugins.base import Plugin
-from couchpotato.core.settings.model import Media
+import six
+
 
 log = CPLog(__name__)
 
@@ -11,35 +15,23 @@ class MediaBase(Plugin):
 
     _type = None
 
-    default_dict = {
-        'profile': {'types': {'quality': {}}},
-        'releases': {'status': {}, 'quality': {}, 'files': {}, 'info': {}},
-        'library': {'titles': {}, 'files': {}},
-        'files': {},
-        'status': {},
-        'category': {},
-    }
-
     def initType(self):
         addEvent('media.types', self.getType)
 
     def getType(self):
         return self._type
 
-    def createOnComplete(self, id):
+    def createOnComplete(self, media_id):
 
         def onComplete():
             try:
-                db = get_session()
-                media = db.query(Media).filter_by(id = id).first()
-                media_dict = media.to_dict(self.default_dict)
-                event_name = '%s.searcher.single' % media.type
+                db = get_db()
+                media = fireEvent('media.get', media_id, single = True)
+                event_name = '%s.searcher.single' % media.get('type')
 
-                fireEvent(event_name, media_dict, on_complete = self.createNotifyFront(id))
+                fireEventAsync(event_name, media, on_complete = self.createNotifyFront(media_id))
             except:
                 log.error('Failed creating onComplete: %s', traceback.format_exc())
-            finally:
-                db.close()
 
         return onComplete
 
@@ -47,15 +39,61 @@ class MediaBase(Plugin):
 
         def notifyFront():
             try:
-                db = get_session()
-                media = db.query(Media).filter_by(id = media_id).first()
-                media_dict = media.to_dict(self.default_dict)
-                event_name = '%s.update' % media.type
+                media = fireEvent('media.get', media_id, single = True)
+                event_name = '%s.update' % media.get('type')
 
-                fireEvent('notify.frontend', type = event_name, data = media_dict)
+                fireEvent('notify.frontend', type = event_name, data = media)
             except:
                 log.error('Failed creating onComplete: %s', traceback.format_exc())
-            finally:
-                db.close()
 
         return notifyFront
+
+    def getDefaultTitle(self, info, ):
+
+        # Set default title
+        default_title = toUnicode(info.get('title'))
+        titles = info.get('titles', [])
+        counter = 0
+        def_title = None
+        for title in titles:
+            if (len(default_title) == 0 and counter == 0) or len(titles) == 1 or title.lower() == toUnicode(default_title.lower()) or (toUnicode(default_title) == six.u('') and toUnicode(titles[0]) == title):
+                def_title = toUnicode(title)
+                break
+            counter += 1
+
+        if not def_title:
+            def_title = toUnicode(titles[0])
+
+        return def_title or 'UNKNOWN'
+
+    def getPoster(self, image_urls, existing_files):
+        image_type = 'poster'
+
+        # Remove non-existing files
+        file_type = 'image_%s' % image_type
+
+        # Make existing unique
+        unique_files = list(set(existing_files.get(file_type, [])))
+
+        # Remove files that can't be found
+        for ef in unique_files:
+            if not os.path.isfile(ef):
+                unique_files.remove(ef)
+
+        # Replace new files list
+        existing_files[file_type] = unique_files
+        if len(existing_files) == 0:
+            del existing_files[file_type]
+
+        # Loop over type
+        for image in image_urls.get(image_type, []):
+            if not isinstance(image, (str, unicode)):
+                continue
+
+            if file_type not in existing_files or len(existing_files.get(file_type, [])) == 0:
+                file_path = fireEvent('file.download', url = image, single = True)
+                if file_path:
+                    existing_files[file_type] = [file_path]
+                    break
+            else:
+                break

couchpotato/core/media/_base/library/__init__.py

@@ -1,13 +1,7 @@
-from couchpotato.core.event import addEvent
-from couchpotato.core.plugins.base import Plugin
+from .main import Library
 
 
-class LibraryBase(Plugin):
+def autoload():
+    return Library()
 
-    _type = None
-
-    def initType(self):
-        addEvent('library.types', self.getType)
-
-    def getType(self):
-        return self._type
+config = []

couchpotato/core/media/_base/library/base.py

@@ -0,0 +1,13 @@
+from couchpotato.core.event import addEvent
+from couchpotato.core.plugins.base import Plugin
+
+
+class LibraryBase(Plugin):
+
+    _type = None
+
+    def initType(self):
+        addEvent('library.types', self.getType)
+
+    def getType(self):
+        return self._type

couchpotato/core/media/_base/library/main.py

@@ -0,0 +1,18 @@
+from couchpotato.core.event import addEvent, fireEvent
+from couchpotato.core.media._base.library.base import LibraryBase
+
+
+class Library(LibraryBase):
+    def __init__(self):
+        addEvent('library.title', self.title)
+
+    def title(self, library):
+        return fireEvent(
+            'library.query',
+            library,
+
+            condense = False,
+            include_year = False,
+            include_identifier = False,
+            single = True
+        )

couchpotato/core/media/_base/matcher/__init__.py

@@ -0,0 +1,7 @@
+from .main import Matcher
+
+
+def autoload():
+    return Matcher()
+
+config = []

couchpotato/core/media/_base/matcher/base.py

@@ -0,0 +1,84 @@
+from couchpotato.core.event import addEvent
+from couchpotato.core.helpers.encoding import simplifyString
+from couchpotato.core.logger import CPLog
+from couchpotato.core.plugins.base import Plugin
+
+log = CPLog(__name__)
+
+
+class MatcherBase(Plugin):
+    type = None
+
+    def __init__(self):
+        if self.type:
+            addEvent('%s.matcher.correct' % self.type, self.correct)
+
+    def correct(self, chain, release, media, quality):
+        raise NotImplementedError()
+
+    def flattenInfo(self, info):
+        # Flatten dictionary of matches (chain info)
+        if isinstance(info, dict):
+            return dict([(key, self.flattenInfo(value)) for key, value in info.items()])
+
+        # Flatten matches
+        result = None
+
+        for match in info:
+            if isinstance(match, dict):
+                if result is None:
+                    result = {}
+
+                for key, value in match.items():
+                    if key not in result:
+                        result[key] = []
+
+                    result[key].append(value)
+            else:
+                if result is None:
+                    result = []
+
+                result.append(match)
+
+        return result
+
+    def constructFromRaw(self, match):
+        if not match:
+            return None
+
+        parts = [
+            ''.join([
+                y for y in x[1:] if y
+            ]) for x in match
+        ]
+
+        return ''.join(parts)[:-1].strip()
+
+    def simplifyValue(self, value):
+        if not value:
+            return value
+
+        if isinstance(value, basestring):
+            return simplifyString(value)
+
+        if isinstance(value, list):
+            return [self.simplifyValue(x) for x in value]
+
+        raise ValueError("Unsupported value type")
+
+    def chainMatch(self, chain, group, tags):
+        info = self.flattenInfo(chain.info[group])
+
+        found_tags = []
+        for tag, accepted in tags.items():
+            values = [self.simplifyValue(x) for x in info.get(tag, [None])]
+
+            if any([val in accepted for val in values]):
+                found_tags.append(tag)
+
+        log.debug('tags found: %s, required: %s' % (found_tags, tags.keys()))
+
+        if set(tags.keys()) == set(found_tags):
+            return True
+
+        return all([key in found_tags for key, value in tags.items()])

couchpotato/core/media/_base/matcher/main.py

@@ -0,0 +1,89 @@
+from couchpotato.core.event import addEvent, fireEvent
+from couchpotato.core.helpers.variable import possibleTitles
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.matcher.base import MatcherBase
+from caper import Caper
+
+log = CPLog(__name__)
+
+
+class Matcher(MatcherBase):
+
+    def __init__(self):
+        super(Matcher, self).__init__()
+
+        self.caper = Caper()
+
+        addEvent('matcher.parse', self.parse)
+        addEvent('matcher.match', self.match)
+
+        addEvent('matcher.flatten_info', self.flattenInfo)
+        addEvent('matcher.construct_from_raw', self.constructFromRaw)
+
+        addEvent('matcher.correct_title', self.correctTitle)
+        addEvent('matcher.correct_quality', self.correctQuality)
+
+    def parse(self, name, parser='scene'):
+        return self.caper.parse(name, parser)
+
+    def match(self, release, media, quality):
+        match = fireEvent('matcher.parse', release['name'], single = True)
+
+        if len(match.chains) < 1:
+            log.info2('Wrong: %s, unable to parse release name (no chains)', release['name'])
+            return False
+
+        for chain in match.chains:
+            if fireEvent('%s.matcher.correct' % media['type'], chain, release, media, quality, single = True):
+                return chain
+
+        return False
+
+    def correctTitle(self, chain, media):
+        root_library = media['library']['root_library']
+
+        if 'show_name' not in chain.info or not len(chain.info['show_name']):
+            log.info('Wrong: missing show name in parsed result')
+            return False
+
+        # Get the lower-case parsed show name from the chain
+        chain_words = [x.lower() for x in chain.info['show_name']]
+
+        # Build a list of possible titles of the media we are searching for
+        titles = root_library['info']['titles']
+
+        # Add year suffix titles (will result in ['<name_one>', '<name_one> <suffix_one>', '<name_two>', ...])
+        suffixes = [None, root_library['info']['year']]
+
+        titles = [
+            title + ((' %s' % suffix) if suffix else '')
+            for title in titles
+            for suffix in suffixes
+        ]
+
+        # Check show titles match
+        # TODO check xem names
+        for title in titles:
+            for valid_words in [x.split(' ') for x in possibleTitles(title)]:
+
+                if valid_words == chain_words:
+                    return True
+
+        return False
+
+    def correctQuality(self, chain, quality, quality_map):
+        if quality['identifier'] not in quality_map:
+            log.info2('Wrong: unknown preferred quality %s', quality['identifier'])
+            return False
+
+        if 'video' not in chain.info:
+            log.info2('Wrong: no video tags found')
+            return False
+
+        video_tags = quality_map[quality['identifier']]
+
+        if not self.chainMatch(chain, 'video', video_tags):
+            log.info2('Wrong: %s tags not in chain', video_tags)
+            return False
+
+        return True

couchpotato/core/media/_base/media/__init__.py

@@ -1,7 +1,5 @@
 from .main import MediaPlugin
 
 
-def start():
+def autoload():
     return MediaPlugin()
-
-config = []

couchpotato/core/media/_base/media/index.py

@@ -0,0 +1,178 @@
+from string import ascii_letters
+from hashlib import md5
+
+from CodernityDB.tree_index import MultiTreeBasedIndex, TreeBasedIndex
+from couchpotato.core.helpers.encoding import toUnicode, simplifyString
+
+
+class MediaIndex(MultiTreeBasedIndex):
+    _version = 3
+
+    custom_header = """from CodernityDB.tree_index import MultiTreeBasedIndex"""
+
+    def __init__(self, *args, **kwargs):
+        kwargs['key_format'] = '32s'
+        super(MediaIndex, self).__init__(*args, **kwargs)
+
+    def make_key(self, key):
+        return md5(key).hexdigest()
+
+    def make_key_value(self, data):
+        if data.get('_t') == 'media' and (data.get('identifier') or data.get('identifiers')):
+
+            identifiers = data.get('identifiers', {})
+            if data.get('identifier') and 'imdb' not in identifiers:
+                identifiers['imdb'] = data.get('identifier')
+
+            ids = []
+            for x in identifiers:
+                ids.append(md5('%s-%s' % (x, identifiers[x])).hexdigest())
+
+            return ids, None
+
+
+class MediaStatusIndex(TreeBasedIndex):
+    _version = 1
+
+    def __init__(self, *args, **kwargs):
+        kwargs['key_format'] = '32s'
+        super(MediaStatusIndex, self).__init__(*args, **kwargs)
+
+    def make_key(self, key):
+        return md5(key).hexdigest()
+
+    def make_key_value(self, data):
+        if data.get('_t') == 'media' and data.get('status'):
+            return md5(data.get('status')).hexdigest(), None
+
+
+class MediaTypeIndex(TreeBasedIndex):
+    _version = 1
+
+    def __init__(self, *args, **kwargs):
+        kwargs['key_format'] = '32s'
+        super(MediaTypeIndex, self).__init__(*args, **kwargs)
+
+    def make_key(self, key):
+        return md5(key).hexdigest()
+
+    def make_key_value(self, data):
+        if data.get('_t') == 'media' and data.get('type'):
+            return md5(data.get('type')).hexdigest(), None
+
+
+class TitleSearchIndex(MultiTreeBasedIndex):
+    _version = 1
+
+    custom_header = """from CodernityDB.tree_index import MultiTreeBasedIndex
+from itertools import izip
+from couchpotato.core.helpers.encoding import simplifyString"""
+
+    def __init__(self, *args, **kwargs):
+        kwargs['key_format'] = '32s'
+        super(TitleSearchIndex, self).__init__(*args, **kwargs)
+        self.__l = kwargs.get('w_len', 2)
+
+    def make_key_value(self, data):
+
+        if data.get('_t') == 'media' and len(data.get('title', '')) > 0:
+
+            out = set()
+            title = str(simplifyString(data.get('title').lower()))
+            l = self.__l
+            title_split = title.split()
+
+            for x in range(len(title_split)):
+                combo = ' '.join(title_split[x:])[:32].strip()
+                out.add(combo.rjust(32, '_'))
+                combo_range = max(l, min(len(combo), 32))
+
+                for cx in range(1, combo_range):
+                    ccombo = combo[:-cx].strip()
+                    if len(ccombo) > l:
+                        out.add(ccombo.rjust(32, '_'))
+
+            return out, None
+
+    def make_key(self, key):
+        return key.rjust(32, '_').lower()
+
+
+class TitleIndex(TreeBasedIndex):
+    _version = 2
+
+    custom_header = """from CodernityDB.tree_index import TreeBasedIndex
+from string import ascii_letters
+from couchpotato.core.helpers.encoding import toUnicode, simplifyString"""
+
+    def __init__(self, *args, **kwargs):
+        kwargs['key_format'] = '32s'
+        super(TitleIndex, self).__init__(*args, **kwargs)
+
+    def make_key(self, key):
+        return self.simplify(key)
+
+    def make_key_value(self, data):
+        if data.get('_t') == 'media' and data.get('title') is not None and len(data.get('title')) > 0:
+            return self.simplify(data['title']), None
+
+    def simplify(self, title):
+
+        title = toUnicode(title)
+
+        nr_prefix = '' if title and len(title) > 0 and title[0] in ascii_letters else '#'
+        title = simplifyString(title)
+
+        for prefix in ['the ']:
+            if prefix == title[:len(prefix)]:
+                title = title[len(prefix):]
+                break
+
+        return str(nr_prefix + title).ljust(32, '_')[:32]
+
+
+class StartsWithIndex(TreeBasedIndex):
+    _version = 2
+
+    custom_header = """from CodernityDB.tree_index import TreeBasedIndex
+from string import ascii_letters
+from couchpotato.core.helpers.encoding import toUnicode, simplifyString"""
+
+    def __init__(self, *args, **kwargs):
+        kwargs['key_format'] = '1s'
+        super(StartsWithIndex, self).__init__(*args, **kwargs)
+
+    def make_key(self, key):
+        return self.first(key)
+
+    def make_key_value(self, data):
+        if data.get('_t') == 'media' and data.get('title') is not None:
+            return self.first(data['title']), None
+
+    def first(self, title):
+        title = toUnicode(title)
+        title = simplifyString(title)
+
+        for prefix in ['the ']:
+            if prefix == title[:len(prefix)]:
+                title = title[len(prefix):]
+                break
+
+        return str(title[0] if title and len(title) > 0 and title[0] in ascii_letters else '#').lower()
+
+
+
+class MediaChildrenIndex(TreeBasedIndex):
+    _version = 1
+
+    def __init__(self, *args, **kwargs):
+        kwargs['key_format'] = '32s'
+        super(MediaChildrenIndex, self).__init__(*args, **kwargs)
+
+    def make_key(self, key):
+        return key
+
+    def make_key_value(self, data):
+        if data.get('_t') == 'media' and data.get('parent_id'):
+            return data.get('parent_id'), None
+

couchpotato/core/media/_base/media/main.py

@@ -1,22 +1,31 @@
 import traceback
-from couchpotato import get_session, tryInt
+from string import ascii_lowercase
+
+from couchpotato import tryInt, get_db
 from couchpotato.api import addApiView
 from couchpotato.core.event import fireEvent, fireEventAsync, addEvent
 from couchpotato.core.helpers.encoding import toUnicode
-from couchpotato.core.helpers.variable import mergeDicts, splitString, getImdb, getTitle
+from couchpotato.core.helpers.variable import splitString, getImdb, getTitle
 from couchpotato.core.logger import CPLog
 from couchpotato.core.media import MediaBase
-from couchpotato.core.settings.model import Library, LibraryTitle, Release, \
-    Media
-from sqlalchemy.orm import joinedload_all
-from sqlalchemy.sql.expression import or_, asc, not_, desc
-from string import ascii_lowercase
+from .index import MediaIndex, MediaStatusIndex, MediaTypeIndex, TitleSearchIndex, TitleIndex, StartsWithIndex, MediaChildrenIndex
+
 
 log = CPLog(__name__)
 
 
 class MediaPlugin(MediaBase):
 
+    _database = {
+        'media': MediaIndex,
+        'media_search_title': TitleSearchIndex,
+        'media_status': MediaStatusIndex,
+        'media_by_type': MediaTypeIndex,
+        'media_title': TitleIndex,
+        'media_startswith': StartsWithIndex,
+        'media_children': MediaChildrenIndex,
+    }
+
     def __init__(self):
 
         addApiView('media.refresh', self.refresh, docs = {
@@ -60,12 +69,14 @@ class MediaPlugin(MediaBase):
 
         addApiView('media.available_chars', self.charView)
 
-        addEvent('app.load', self.addSingleRefreshView)
-        addEvent('app.load', self.addSingleListView)
-        addEvent('app.load', self.addSingleCharView)
-        addEvent('app.load', self.addSingleDeleteView)
+        addEvent('app.load', self.addSingleRefreshView, priority = 100)
+        addEvent('app.load', self.addSingleListView, priority = 100)
+        addEvent('app.load', self.addSingleCharView, priority = 100)
+        addEvent('app.load', self.addSingleDeleteView, priority = 100)
 
         addEvent('media.get', self.get)
+        addEvent('media.with_status', self.withStatus)
+        addEvent('media.with_identifiers', self.withIdentifiers)
         addEvent('media.list', self.list)
         addEvent('media.delete', self.delete)
         addEvent('media.restatus', self.restatus)
@@ -80,29 +91,27 @@ class MediaPlugin(MediaBase):
             if refresh_handler:
                 handlers.append(refresh_handler)
 
-        fireEvent('notify.frontend', type = 'media.busy', data = {'id': [tryInt(x) for x in ids]})
+        fireEvent('notify.frontend', type = 'media.busy', data = {'_id': ids})
         fireEventAsync('schedule.queue', handlers = handlers)
 
         return {
             'success': True,
         }
 
-    def createRefreshHandler(self, id):
-        db = get_session()
+    def createRefreshHandler(self, media_id):
 
-        media = db.query(Media).filter_by(id = id).first()
-
-        if media:
-
-            default_title = getTitle(media.library)
-            identifier = media.library.identifier
-            event = 'library.update.%s' % media.type
+        try:
+            media = get_db().get('id', media_id)
+            event = '%s.update_info' % media.get('type')
 
             def handler():
-                fireEvent(event, identifier = identifier, default_title = default_title, on_complete = self.createOnComplete(id))
+                fireEvent(event, media_id = media_id, on_complete = self.createOnComplete(media_id))
 
-        if handler:
-            return handler
+            if handler:
+                return handler
+
+        except:
+            log.error('Refresh handler for non existing media: %s', traceback.format_exc())
 
     def addSingleRefreshView(self):
 
@@ -111,20 +120,25 @@ class MediaPlugin(MediaBase):
 
     def get(self, media_id):
 
-        db = get_session()
+        db = get_db()
 
         imdb_id = getImdb(str(media_id))
 
+        media = None
         if imdb_id:
-            m = db.query(Media).filter(Media.library.has(identifier = imdb_id)).first()
+            media = db.get('media', 'imdb-%s' % imdb_id, with_doc = True)['doc']
         else:
-            m = db.query(Media).filter_by(id = media_id).first()
+            media = db.get('id', media_id)
 
-        results = None
-        if m:
-            results = m.to_dict(self.default_dict)
+        if media:
 
-        return results
+            # Attach category
+            try: media['category'] = db.get('id', media.get('category_id'))
+            except: pass
+
+            media['releases'] = fireEvent('release.for_media', media['_id'], single = True)
+
+        return media
 
     def getView(self, id = None, **kwargs):
 
@@ -135,9 +149,32 @@ class MediaPlugin(MediaBase):
             'media': media,
         }
 
-    def list(self, types = None, status = None, release_status = None, limit_offset = None, starts_with = None, search = None, order = None):
+    def withStatus(self, status, with_doc = True):
 
-        db = get_session()
+        db = get_db()
+
+        status = list(status if isinstance(status, (list, tuple)) else [status])
+
+        for s in status:
+            for ms in db.get_many('media_status', s, with_doc = with_doc):
+                yield ms['doc'] if with_doc else ms
+
+    def withIdentifiers(self, identifiers, with_doc = False):
+
+        db = get_db()
+
+        for x in identifiers:
+            try:
+                media = db.get('media', '%s-%s' % (x, identifiers[x]), with_doc = with_doc)
+                return media
+            except:
+                pass
+
+        log.debug('No media found with identifiers: %s', identifiers)
+
+    def list(self, types = None, status = None, release_status = None, status_or = False, limit_offset = None, starts_with = None, search = None):
+
+        db = get_db()
 
         # Make a list from string
         if status and not isinstance(status, (list, tuple)):
@@ -147,138 +184,90 @@ class MediaPlugin(MediaBase):
         if types and not isinstance(types, (list, tuple)):
             types = [types]
 
-        # query movie ids
-        q = db.query(Media) \
-            .with_entities(Media.id) \
-            .group_by(Media.id)
+        # query media ids
+        if types:
+            all_media_ids = set()
+            for media_type in types:
+                all_media_ids = all_media_ids.union(set([x['_id'] for x in db.get_many('media_by_type', media_type)]))
+        else:
+            all_media_ids = set([x['_id'] for x in db.all('media')])
+
+        media_ids = list(all_media_ids)
+        filter_by = {}
 
         # Filter on movie status
         if status and len(status) > 0:
-            statuses = fireEvent('status.get', status, single = len(status) > 1)
-            statuses = [s.get('id') for s in statuses]
-
-            q = q.filter(Media.status_id.in_(statuses))
+            filter_by['media_status'] = set()
+            for media_status in fireEvent('media.with_status', status, with_doc = False, single = True):
+                filter_by['media_status'].add(media_status.get('_id'))
 
         # Filter on release status
         if release_status and len(release_status) > 0:
-            q = q.join(Media.releases)
-
-            statuses = fireEvent('status.get', release_status, single = len(release_status) > 1)
-            statuses = [s.get('id') for s in statuses]
-
-            q = q.filter(Release.status_id.in_(statuses))
-
-        # Filter on type
-        if types and len(types) > 0:
-            try: q = q.filter(Media.type.in_(types))
-            except: pass
-
-        # Only join when searching / ordering
-        if starts_with or search or order != 'release_order':
-            q = q.join(Media.library, Library.titles) \
-                .filter(LibraryTitle.default == True)
+            filter_by['release_status'] = set()
+            for release_status in fireEvent('release.with_status', release_status, with_doc = False, single = True):
+                filter_by['release_status'].add(release_status.get('media_id'))
 
         # Add search filters
-        filter_or = []
         if starts_with:
-            starts_with = toUnicode(starts_with.lower())
-            if starts_with in ascii_lowercase:
-                filter_or.append(LibraryTitle.simple_title.startswith(starts_with))
-            else:
-                ignore = []
-                for letter in ascii_lowercase:
-                    ignore.append(LibraryTitle.simple_title.startswith(toUnicode(letter)))
-                filter_or.append(not_(or_(*ignore)))
+            filter_by['starts_with'] = set()
+            starts_with = toUnicode(starts_with.lower())[0]
+            starts_with = starts_with if starts_with in ascii_lowercase else '#'
+            filter_by['starts_with'] = [x['_id'] for x in db.get_many('media_startswith', starts_with)]
 
+        # Filter with search query
         if search:
-            filter_or.append(LibraryTitle.simple_title.like('%%' + search + '%%'))
+            filter_by['search'] = [x['_id'] for x in db.get_many('media_search_title', search)]
 
-        if len(filter_or) > 0:
-            q = q.filter(or_(*filter_or))
+        if status_or and 'media_status' in filter_by and 'release_status' in filter_by:
+            filter_by['status'] = list(filter_by['media_status']) + list(filter_by['release_status'])
+            del filter_by['media_status']
+            del filter_by['release_status']
 
-        total_count = q.count()
+        # Filter by combining ids
+        for x in filter_by:
+            media_ids = [n for n in media_ids if n in filter_by[x]]
+
+        total_count = len(media_ids)
         if total_count == 0:
             return 0, []
 
-        if order == 'release_order':
-            q = q.order_by(desc(Release.last_edit))
-        else:
-            q = q.order_by(asc(LibraryTitle.simple_title))
-
+        offset = 0
+        limit = -1
         if limit_offset:
             splt = splitString(limit_offset) if isinstance(limit_offset, (str, unicode)) else limit_offset
-            limit = splt[0]
-            offset = 0 if len(splt) is 1 else splt[1]
-            q = q.limit(limit).offset(offset)
+            limit = tryInt(splt[0])
+            offset = tryInt(0 if len(splt) is 1 else splt[1])
 
-        # Get all media_ids in sorted order
-        media_ids = [m.id for m in q.all()]
+        # List movies based on title order
+        medias = []
+        for m in db.all('media_title'):
+            media_id = m['_id']
+            if media_id not in media_ids: continue
+            if offset > 0:
+                offset -= 1
+                continue
 
-        # List release statuses
-        releases = db.query(Release) \
-            .filter(Release.movie_id.in_(media_ids)) \
-            .all()
-
-        release_statuses = dict((m, set()) for m in media_ids)
-        releases_count = dict((m, 0) for m in media_ids)
-        for release in releases:
-            release_statuses[release.movie_id].add('%d,%d' % (release.status_id, release.quality_id))
-            releases_count[release.movie_id] += 1
-
-        # Get main movie data
-        q2 = db.query(Media) \
-            .options(joinedload_all('library.titles')) \
-            .options(joinedload_all('library.files')) \
-            .options(joinedload_all('status')) \
-            .options(joinedload_all('files'))
-
-        q2 = q2.filter(Media.id.in_(media_ids))
-
-        results = q2.all()
-
-        # Create dict by movie id
-        movie_dict = {}
-        for movie in results:
-            movie_dict[movie.id] = movie
-
-        # List movies based on media_ids order
-        movies = []
-        for media_id in media_ids:
-
-            releases = []
-            for r in release_statuses.get(media_id):
-                x = splitString(r)
-                releases.append({'status_id': x[0], 'quality_id': x[1]})
+            media = fireEvent('media.get', media_id, single = True)
 
             # Merge releases with movie dict
-            movies.append(mergeDicts(movie_dict[media_id].to_dict({
-                'library': {'titles': {}, 'files': {}},
-                'files': {},
-            }), {
-                'releases': releases,
-                'releases_count': releases_count.get(media_id),
-            }))
+            medias.append(media)
 
-        return total_count, movies
+            # remove from media ids
+            media_ids.remove(media_id)
+            if len(media_ids) == 0 or len(medias) == limit: break
+
+        return total_count, medias
 
     def listView(self, **kwargs):
 
-        types = splitString(kwargs.get('types'))
-        status = splitString(kwargs.get('status'))
-        release_status = splitString(kwargs.get('release_status'))
-        limit_offset = kwargs.get('limit_offset')
-        starts_with = kwargs.get('starts_with')
-        search = kwargs.get('search')
-        order = kwargs.get('order')
-
         total_movies, movies = self.list(
-            types = types,
-            status = status,
-            release_status = release_status,
-            limit_offset = limit_offset,
-            starts_with = starts_with,
-            search = search,
-            order = order
+            types = splitString(kwargs.get('type')),
+            status = splitString(kwargs.get('status')),
+            release_status = splitString(kwargs.get('release_status')),
+            status_or = kwargs.get('status_or') is not None,
+            limit_offset = kwargs.get('limit_offset'),
+            starts_with = kwargs.get('starts_with'),
+            search = kwargs.get('search')
         )
 
         return {
@@ -292,67 +281,57 @@ class MediaPlugin(MediaBase):
 
         for media_type in fireEvent('media.types', merge = True):
             def tempList(*args, **kwargs):
-                return self.listView(types = media_type, *args, **kwargs)
+                return self.listView(types = media_type, **kwargs)
             addApiView('%s.list' % media_type, tempList)
 
     def availableChars(self, types = None, status = None, release_status = None):
 
-        types = types or []
-        status = status or []
-        release_status = release_status or []
-
-        db = get_session()
+        db = get_db()
 
         # Make a list from string
-        if not isinstance(status, (list, tuple)):
+        if status and not isinstance(status, (list, tuple)):
             status = [status]
         if release_status and not isinstance(release_status, (list, tuple)):
             release_status = [release_status]
         if types and not isinstance(types, (list, tuple)):
             types = [types]
 
-        q = db.query(Media)
+        # query media ids
+        if types:
+            all_media_ids = set()
+            for media_type in types:
+                all_media_ids = all_media_ids.union(set([x['_id'] for x in db.get_many('media_by_type', media_type)]))
+        else:
+            all_media_ids = set([x['_id'] for x in db.all('media')])
+
+        media_ids = all_media_ids
+        filter_by = {}
 
         # Filter on movie status
         if status and len(status) > 0:
-            statuses = fireEvent('status.get', status, single = len(release_status) > 1)
-            statuses = [s.get('id') for s in statuses]
-
-            q = q.filter(Media.status_id.in_(statuses))
+            filter_by['media_status'] = set()
+            for media_status in fireEvent('media.with_status', status, with_doc = False, single = True):
+                filter_by['media_status'].add(media_status.get('_id'))
 
         # Filter on release status
         if release_status and len(release_status) > 0:
+            filter_by['release_status'] = set()
+            for release_status in fireEvent('release.with_status', release_status, with_doc = False, single = True):
+                filter_by['release_status'].add(release_status.get('media_id'))
 
-            statuses = fireEvent('status.get', release_status, single = len(release_status) > 1)
-            statuses = [s.get('id') for s in statuses]
-
-            q = q.join(Media.releases) \
-                .filter(Release.status_id.in_(statuses))
-
-        # Filter on type
-        if types and len(types) > 0:
-            try: q = q.filter(Media.type.in_(types))
-            except: pass
-
-        q = q.join(Library, LibraryTitle) \
-            .with_entities(LibraryTitle.simple_title) \
-            .filter(LibraryTitle.default == True)
-
-        titles = q.all()
+        # Filter by combining ids
+        for x in filter_by:
+            media_ids = [n for n in media_ids if n in filter_by[x]]
 
         chars = set()
-        for title in titles:
-            try:
-                char = title[0][0]
-                char = char if char in ascii_lowercase else '#'
-                chars.add(str(char))
-            except:
-                log.error('Failed getting title for %s', title.libraries_id)
+        for x in db.all('media_startswith'):
+            if x['_id'] in media_ids:
+                chars.add(x['key'])
 
             if len(chars) == 25:
                 break
 
-        return ''.join(sorted(chars))
+        return list(chars)
 
     def charView(self, **kwargs):
 
@@ -371,59 +350,52 @@ class MediaPlugin(MediaBase):
 
         for media_type in fireEvent('media.types', merge = True):
             def tempChar(*args, **kwargs):
-                return self.charView(types = media_type, *args, **kwargs)
+                return self.charView(types = media_type, **kwargs)
             addApiView('%s.available_chars' % media_type, tempChar)
 
     def delete(self, media_id, delete_from = None):
 
         try:
-            db = get_session()
+            db = get_db()
 
-            media = db.query(Media).filter_by(id = media_id).first()
+            media = db.get('id', media_id)
             if media:
                 deleted = False
                 if delete_from == 'all':
                     db.delete(media)
-                    db.commit()
                     deleted = True
                 else:
-                    done_status = fireEvent('status.get', 'done', single = True)
 
-                    total_releases = len(media.releases)
+                    media_releases = fireEvent('release.for_media', media['_id'], single = True)
+
+                    total_releases = len(media_releases)
                     total_deleted = 0
-                    new_movie_status = None
-                    for release in media.releases:
-                        if delete_from in ['wanted', 'snatched', 'late']:
-                            if release.status_id != done_status.get('id'):
-                                db.delete(release)
-                                total_deleted += 1
-                            new_movie_status = 'done'
-                        elif delete_from == 'manage':
-                            if release.status_id == done_status.get('id'):
-                                db.delete(release)
-                                total_deleted += 1
-                            new_movie_status = 'active'
-                    db.commit()
+                    new_media_status = None
 
-                    if total_releases == total_deleted:
+                    for release in media_releases:
+                        if delete_from in ['wanted', 'snatched', 'late']:
+                            if release.get('status') != 'done':
+                                db.delete(release)
+                                total_deleted += 1
+                            new_media_status = 'done'
+                        elif delete_from == 'manage':
+                            if release.get('status') == 'done':
+                                db.delete(release)
+                                total_deleted += 1
+
+                    if (total_releases == total_deleted and media['status'] != 'active') or (delete_from == 'wanted' and media['status'] == 'active'):
                         db.delete(media)
-                        db.commit()
                         deleted = True
-                    elif new_movie_status:
-                        new_status = fireEvent('status.get', new_movie_status, single = True)
-                        media.profile_id = None
-                        media.status_id = new_status.get('id')
-                        db.commit()
+                    elif new_media_status:
+                        media['status'] = new_media_status
+                        db.update(media)
                     else:
-                        fireEvent('media.restatus', media.id, single = True)
+                        fireEvent('media.restatus', media.get('_id'), single = True)
 
                 if deleted:
-                    fireEvent('notify.frontend', type = 'movie.deleted', data = media.to_dict())
+                    fireEvent('notify.frontend', type = 'media.deleted', data = media)
         except:
             log.error('Failed deleting media: %s', traceback.format_exc())
-            db.rollback()
-        finally:
-            db.close()
 
         return True
 
@@ -446,35 +418,34 @@ class MediaPlugin(MediaBase):
 
     def restatus(self, media_id):
 
-        active_status, done_status = fireEvent('status.get', ['active', 'done'], single = True)
-
         try:
-            db = get_session()
+            db = get_db()
 
-            m = db.query(Media).filter_by(id = media_id).first()
-            if not m or len(m.library.titles) == 0:
-                log.debug('Can\'t restatus movie, doesn\'t seem to exist.')
-                return False
+            m = db.get('id', media_id)
+            previous_status = m['status']
 
-            log.debug('Changing status for %s', m.library.titles[0].title)
-            if not m.profile:
-                m.status_id = done_status.get('id')
+            log.debug('Changing status for %s', getTitle(m))
+            if not m['profile_id']:
+                m['status'] = 'done'
             else:
                 move_to_wanted = True
 
-                for t in m.profile.types:
-                    for release in m.releases:
-                        if t.quality.identifier is release.quality.identifier and (release.status_id is done_status.get('id') and t.finish):
+                profile = db.get('id', m['profile_id'])
+                media_releases = fireEvent('release.for_media', m['_id'], single = True)
+
+                for q_identifier in profile['qualities']:
+                    index = profile['qualities'].index(q_identifier)
+
+                    for release in media_releases:
+                        if q_identifier == release['quality'] and (release.get('status') == 'done' and profile['finish'][index]):
                             move_to_wanted = False
 
-                m.status_id = active_status.get('id') if move_to_wanted else done_status.get('id')
+                m['status'] = 'active' if move_to_wanted else 'done'
 
-            db.commit()
+            # Only update when status has changed
+            if previous_status != m['status']:
+                db.update(m)
 
             return True
         except:
             log.error('Failed restatus: %s', traceback.format_exc())
-            db.rollback()
-        finally:
-            db.close()
-

couchpotato/core/media/_base/providers/automation/base.py

@@ -0,0 +1,8 @@
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.base import Provider
+
+log = CPLog(__name__)
+
+
+class AutomationBase(Provider):
+    pass

couchpotato/core/media/_base/providers/base.py

@@ -1,10 +1,3 @@
-from couchpotato.core.event import addEvent, fireEvent
-from couchpotato.core.helpers.encoding import ss
-from couchpotato.core.helpers.variable import tryFloat, mergeDicts, md5, \
-    possibleTitles, getTitle
-from couchpotato.core.logger import CPLog
-from couchpotato.core.plugins.base import Plugin
-from couchpotato.environment import Env
 from urlparse import urlparse
 import json
 import re
@@ -12,6 +5,15 @@ import time
 import traceback
 import xml.etree.ElementTree as XMLTree
 
+from couchpotato.core.event import addEvent, fireEvent
+from couchpotato.core.helpers.encoding import ss
+from couchpotato.core.helpers.variable import tryFloat, mergeDicts, md5, \
+    possibleTitles
+from couchpotato.core.logger import CPLog
+from couchpotato.core.plugins.base import Plugin
+from couchpotato.environment import Env
+
+
 log = CPLog(__name__)
 
 
@@ -102,7 +104,6 @@ class Provider(Plugin):
 class YarrProvider(Provider):
 
     protocol = None  # nzb, torrent, torrent_magnet
-    type = 'movie'
 
     cat_ids = {}
     cat_backup_id = None
@@ -180,7 +181,7 @@ class YarrProvider(Provider):
 
         return 'try_next'
 
-    def search(self, movie, quality):
+    def search(self, media, quality):
 
         if self.isDisabled():
             return []
@@ -192,15 +193,17 @@ class YarrProvider(Provider):
 
         # Create result container
         imdb_results = hasattr(self, '_search')
-        results = ResultList(self, movie, quality, imdb_results = imdb_results)
+        results = ResultList(self, media, quality, imdb_results = imdb_results)
 
         # Do search based on imdb id
         if imdb_results:
-            self._search(movie, quality, results)
+            self._search(media, quality, results)
         # Search possible titles
         else:
-            for title in possibleTitles(getTitle(movie['library'])):
-                self._searchOnTitle(title, movie, quality, results)
+            media_title = fireEvent('library.query', media, single = True)
+
+            for title in possibleTitles(media_title):
+                self._searchOnTitle(title, media, quality, results)
 
         return results
 
@@ -241,11 +244,16 @@ class YarrProvider(Provider):
 
         return 0
 
-    def getCatId(self, identifier):
+    def getCatId(self, quality = None):
+        if not quality: quality = {}
+        identifier = quality.get('identifier')
 
-        for cats in self.cat_ids:
-            ids, qualities = cats
-            if identifier in qualities:
+        want_3d = False
+        if quality.get('custom'):
+            want_3d = quality['custom'].get('3d')
+
+        for ids, qualities in self.cat_ids:
+            if identifier in qualities or (want_3d and '3d' in qualities):
                 return ids
 
         if self.cat_backup_id:

couchpotato/core/media/_base/providers/info/base.py

@@ -0,0 +1,5 @@
+from couchpotato.core.media._base.providers.base import Provider
+
+
+class BaseInfoProvider(Provider):
+    type = 'unknown'

couchpotato/core/media/_base/providers/metadata/base.py

@@ -0,0 +1,8 @@
+from couchpotato.core.logger import CPLog
+from couchpotato.core.plugins.base import Plugin
+
+log = CPLog(__name__)
+
+
+class MetaDataBase(Plugin):
+    pass

couchpotato/core/media/_base/providers/nzb/base.py

@@ -1,6 +1,7 @@
-from couchpotato.core.providers.base import YarrProvider
 import time
 
+from couchpotato.core.media._base.providers.base import YarrProvider
+
 
 class NZBProvider(YarrProvider):
 

couchpotato/core/media/_base/providers/nzb/binsearch.py

@@ -1,16 +1,16 @@
-from bs4 import BeautifulSoup
-from couchpotato.core.helpers.encoding import tryUrlencode
-from couchpotato.core.helpers.variable import tryInt
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.nzb.base import NZBProvider
-from couchpotato.environment import Env
 import re
 import traceback
 
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.nzb.base import NZBProvider
+
+
 log = CPLog(__name__)
 
 
-class BinSearch(NZBProvider):
+class Base(NZBProvider):
 
     urls = {
         'download': 'https://www.binsearch.info/fcgi/nzb.fcgi?q=%s',
@@ -20,27 +20,15 @@ class BinSearch(NZBProvider):
 
     http_time_between_calls = 4  # Seconds
 
-    def _search(self, movie, quality, results):
+    def _search(self, media, quality, results):
 
-        arguments = tryUrlencode({
-            'q': movie['library']['identifier'],
-            'm': 'n',
-            'max': 400,
-            'adv_age': Env.setting('retention', 'nzb'),
-            'adv_sort': 'date',
-            'adv_col': 'on',
-            'adv_nfo': 'on',
-            'minsize': quality.get('size_min'),
-            'maxsize': quality.get('size_max'),
-        })
-
-        data = self.getHTMLData(self.urls['search'] % arguments)
+        data = self.getHTMLData(self.urls['search'] % self.buildUrl(media, quality))
 
         if data:
             try:
 
                 html = BeautifulSoup(data)
-                main_table = html.find('table', attrs = {'id':'r2'})
+                main_table = html.find('table', attrs = {'id': 'r2'})
 
                 if not main_table:
                     return
@@ -48,11 +36,11 @@ class BinSearch(NZBProvider):
                 items = main_table.find_all('tr')
 
                 for row in items:
-                    title = row.find('span', attrs = {'class':'s'})
+                    title = row.find('span', attrs = {'class': 's'})
 
                     if not title: continue
 
-                    nzb_id = row.find('input', attrs = {'type':'checkbox'})['name']
+                    nzb_id = row.find('input', attrs = {'type': 'checkbox'})['name']
                     info = row.find('span', attrs = {'class':'d'})
                     size_match = re.search('size:.(?P<size>[0-9\.]+.[GMB]+)', info.text)
 
@@ -65,7 +53,7 @@ class BinSearch(NZBProvider):
                         total = tryInt(parts.group('total'))
                         parts = tryInt(parts.group('parts'))
 
-                        if (total / parts) < 0.95 or ((total / parts) >= 0.95 and not ('par2' in info.text.lower() or 'pa3' in info.text.lower())):
+                        if (total / parts) < 1 and ((total / parts) < 0.95 or ((total / parts) >= 0.95 and not ('par2' in info.text.lower() or 'pa3' in info.text.lower()))):
                             log.info2('Wrong: \'%s\', not complete: %s out of %s', (item['name'], parts, total))
                             return False
 
@@ -102,3 +90,30 @@ class BinSearch(NZBProvider):
 
         return 'try_next'
 
+
+config = [{
+    'name': 'binsearch',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'nzb_providers',
+            'name': 'binsearch',
+            'description': 'Free provider, less accurate. See <a href="https://www.binsearch.info/">BinSearch</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/nzb/newznab.py

@@ -1,53 +1,51 @@
-from couchpotato.core.helpers.encoding import tryUrlencode, toUnicode
-from couchpotato.core.helpers.rss import RSS
-from couchpotato.core.helpers.variable import cleanHost, splitString, tryInt
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.base import ResultList
-from couchpotato.core.providers.nzb.base import NZBProvider
-from couchpotato.environment import Env
-from dateutil.parser import parse
 from urllib2 import HTTPError
 from urlparse import urlparse
 import time
 import traceback
 import urllib2
 
+from couchpotato.core.helpers.encoding import tryUrlencode, toUnicode
+from couchpotato.core.helpers.rss import RSS
+from couchpotato.core.helpers.variable import cleanHost, splitString, tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.base import ResultList
+from couchpotato.core.media._base.providers.nzb.base import NZBProvider
+from couchpotato.environment import Env
+from dateutil.parser import parse
+
+
 log = CPLog(__name__)
 
 
-class Newznab(NZBProvider, RSS):
+class Base(NZBProvider, RSS):
 
     urls = {
-        'download': 'get&id=%s',
         'detail': 'details&id=%s',
-        'search': 'movie',
+        'download': 't=get&id=%s'
     }
 
     limits_reached = {}
 
     http_time_between_calls = 1  # Seconds
 
-    def search(self, movie, quality):
+    def search(self, media, quality):
         hosts = self.getHosts()
 
-        results = ResultList(self, movie, quality, imdb_results = True)
+        results = ResultList(self, media, quality, imdb_results = True)
 
         for host in hosts:
             if self.isDisabled(host):
                 continue
 
-            self._searchOnHost(host, movie, quality, results)
+            self._searchOnHost(host, media, quality, results)
 
         return results
 
-    def _searchOnHost(self, host, movie, quality, results):
+    def _searchOnHost(self, host, media, quality, results):
 
-        arguments = tryUrlencode({
-            'imdbid': movie['library']['identifier'].replace('tt', ''),
-            'apikey': host['api_key'],
-            'extended': 1
-        }) + ('&%s' % host['custom_tag'] if host.get('custom_tag') else '')
-        url = '%s&%s' % (self.getUrl(host['host'], self.urls['search']), arguments)
+        query = self.buildUrl(media, host['api_key'])
+
+        url = '%s&%s' % (self.getUrl(host['host']), query)
 
         nzbs = self.getRSSData(url, cache_timeout = 1800, headers = {'User-Agent': Env.getIdentifier()})
 
@@ -88,7 +86,7 @@ class Newznab(NZBProvider, RSS):
                 'name_extra': name_extra,
                 'age': self.calculateAge(int(time.mktime(parse(date).timetuple()))),
                 'size': int(self.getElement(nzb, 'enclosure').attrib['length']) / 1024 / 1024,
-                'url': (self.getUrl(host['host'], self.urls['download']) % tryUrlencode(nzb_id)) + self.getApiExt(host),
+                'url': ((self.getUrl(host['host']) + self.urls['download']) % tryUrlencode(nzb_id)) + self.getApiExt(host),
                 'detail_url': '%sdetails/%s' % (cleanHost(host['host']), tryUrlencode(nzb_id)),
                 'content': self.getTextElement(nzb, 'description'),
                 'score': host['extra_score'],
@@ -132,15 +130,15 @@ class Newznab(NZBProvider, RSS):
         hosts = self.getHosts()
 
         for host in hosts:
-            result = super(Newznab, self).belongsTo(url, host = host['host'], provider = provider)
+            result = super(Base, self).belongsTo(url, host = host['host'], provider = provider)
             if result:
                 return result
 
-    def getUrl(self, host, type):
+    def getUrl(self, host):
         if '?page=newznabapi' in host:
-            return cleanHost(host)[:-1] + '&t=' + type
+            return cleanHost(host)[:-1] + '&'
 
-        return cleanHost(host) + 'api?t=' + type
+        return cleanHost(host) + 'api?'
 
     def isDisabled(self, host = None):
         return not self.isEnabled(host)
@@ -192,3 +190,59 @@ class Newznab(NZBProvider, RSS):
             log.error('Failed download from %s: %s', (host, traceback.format_exc()))
 
         return 'try_next'
+
+
+config = [{
+    'name': 'newznab',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'nzb_providers',
+            'name': 'newznab',
+            'order': 10,
+            'description': 'Enable <a href="http://newznab.com/" target="_blank">NewzNab</a> such as <a href="https://nzb.su" target="_blank">NZB.su</a>, \
+                <a href="https://nzbs.org" target="_blank">NZBs.org</a>, <a href="http://dognzb.cr/" target="_blank">DOGnzb.cr</a>, \
+                <a href="https://github.com/spotweb/spotweb" target="_blank">Spotweb</a>, <a href="https://nzbgeek.info/" target="_blank">NZBGeek</a>, \
+                <a href="https://smackdownonyou.com" target="_blank">SmackDown</a>, <a href="https://www.nzbfinder.ws" target="_blank">NZBFinder</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': True,
+                },
+                {
+                    'name': 'use',
+                    'default': '0,0,0,0,0,0'
+                },
+                {
+                    'name': 'host',
+                    'default': 'api.nzb.su,dognzb.cr,nzbs.org,https://index.nzbgeek.info, https://smackdownonyou.com, https://www.nzbfinder.ws',
+                    'description': 'The hostname of your newznab provider',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'default': '0,0,0,0,0,0',
+                    'description': 'Starting score for each release found via this provider.',
+                },
+                {
+                    'name': 'custom_tag',
+                    'advanced': True,
+                    'label': 'Custom tag',
+                    'default': ',,,,,',
+                    'description': 'Add custom tags, for example add rls=1 to get only scene releases from nzbs.org',
+                },
+                {
+                    'name': 'api_key',
+                    'default': ',,,,,',
+                    'label': 'Api Key',
+                    'description': 'Can be found on your profile page',
+                    'type': 'combined',
+                    'combine': ['use', 'host', 'api_key', 'extra_score', 'custom_tag'],
+                },
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/nzb/nzbclub.py

@@ -1,40 +1,28 @@
+import time
+
 from bs4 import BeautifulSoup
-from couchpotato.core.helpers.encoding import toUnicode, tryUrlencode
+from couchpotato.core.helpers.encoding import toUnicode
 from couchpotato.core.helpers.rss import RSS
 from couchpotato.core.helpers.variable import tryInt
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.nzb.base import NZBProvider
+from couchpotato.core.media._base.providers.nzb.base import NZBProvider
 from dateutil.parser import parse
-import time
+
 
 log = CPLog(__name__)
 
 
-class NZBClub(NZBProvider, RSS):
+class Base(NZBProvider, RSS):
 
     urls = {
         'search': 'https://www.nzbclub.com/nzbfeeds.aspx?%s',
     }
 
-    http_time_between_calls = 4  #seconds
+    http_time_between_calls = 4  # seconds
 
-    def _searchOnTitle(self, title, movie, quality, results):
+    def _search(self, media, quality, results):
 
-        q = '"%s %s"' % (title, movie['library']['year'])
-
-        q_param = tryUrlencode({
-            'q': q,
-        })
-
-        params = tryUrlencode({
-            'ig': 1,
-            'rpp': 200,
-            'st': 5,
-            'sp': 1,
-            'ns': 1,
-        })
-
-        nzbs = self.getRSSData(self.urls['search'] % ('%s&%s' % (q_param, params)))
+        nzbs = self.getRSSData(self.urls['search'] % self.buildUrl(media))
 
         for nzb in nzbs:
 
@@ -67,7 +55,7 @@ class NZBClub(NZBProvider, RSS):
     def getMoreInfo(self, item):
         full_description = self.getCache('nzbclub.%s' % item['id'], item['detail_url'], cache_timeout = 25920000)
         html = BeautifulSoup(full_description)
-        nfo_pre = html.find('pre', attrs = {'class':'nfo'})
+        nfo_pre = html.find('pre', attrs = {'class': 'nfo'})
         description = toUnicode(nfo_pre.text) if nfo_pre else ''
 
         item['description'] = description
@@ -81,3 +69,31 @@ class NZBClub(NZBProvider, RSS):
             return False
 
         return True
+
+
+config = [{
+    'name': 'nzbclub',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'nzb_providers',
+            'name': 'NZBClub',
+            'description': 'Free provider, less accurate. See <a href="https://www.nzbclub.com/">NZBClub</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/nzb/nzbindex.py

@@ -0,0 +1,125 @@
+import re
+import time
+
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.encoding import toUnicode
+from couchpotato.core.helpers.rss import RSS
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.event import fireEvent
+from couchpotato.core.media._base.providers.nzb.base import NZBProvider
+from dateutil.parser import parse
+
+
+log = CPLog(__name__)
+
+
+class Base(NZBProvider, RSS):
+
+    urls = {
+        'download': 'https://www.nzbindex.com/download/',
+        'search': 'https://www.nzbindex.com/rss/?%s',
+    }
+
+    http_time_between_calls = 1  # Seconds
+
+    def _search(self, media, quality, results):
+
+        nzbs = self.getRSSData(self.urls['search'] % self.buildUrl(media, quality))
+
+        for nzb in nzbs:
+
+            enclosure = self.getElement(nzb, 'enclosure').attrib
+            nzbindex_id = int(self.getTextElement(nzb, "link").split('/')[4])
+
+            title = self.getTextElement(nzb, "title")
+
+            match = fireEvent('matcher.parse', title, parser='usenet', single = True)
+            if not match.chains:
+                log.info('Unable to parse release with title "%s"', title)
+                continue
+
+            # TODO should we consider other lower-weight chains here?
+            info = fireEvent('matcher.flatten_info', match.chains[0].info, single = True)
+
+            release_name = fireEvent('matcher.construct_from_raw', info.get('release_name'), single = True)
+
+            file_name = info.get('detail', {}).get('file_name')
+            file_name = file_name[0] if file_name else None
+
+            title = release_name or file_name
+
+            # Strip extension from parsed title (if one exists)
+            ext_pos = title.rfind('.')
+
+            # Assume extension if smaller than 4 characters
+            # TODO this should probably be done a better way
+            if len(title[ext_pos + 1:]) <= 4:
+                title = title[:ext_pos]
+
+            if not title:
+                log.info('Unable to find release name from match')
+                continue
+
+            try:
+                description = self.getTextElement(nzb, "description")
+            except:
+                description = ''
+
+            def extra_check(item):
+                if '#c20000' in item['description'].lower():
+                    log.info('Wrong: Seems to be passworded: %s', item['name'])
+                    return False
+
+                return True
+
+            results.append({
+                'id': nzbindex_id,
+                'name': title,
+                'age': self.calculateAge(int(time.mktime(parse(self.getTextElement(nzb, "pubDate")).timetuple()))),
+                'size': tryInt(enclosure['length']) / 1024 / 1024,
+                'url': enclosure['url'],
+                'detail_url': enclosure['url'].replace('/download/', '/release/'),
+                'description': description,
+                'get_more_info': self.getMoreInfo,
+                'extra_check': extra_check,
+            })
+
+    def getMoreInfo(self, item):
+        try:
+            if '/nfo/' in item['description'].lower():
+                nfo_url = re.search('href=\"(?P<nfo>.+)\" ', item['description']).group('nfo')
+                full_description = self.getCache('nzbindex.%s' % item['id'], url = nfo_url, cache_timeout = 25920000)
+                html = BeautifulSoup(full_description)
+                item['description'] = toUnicode(html.find('pre', attrs = {'id': 'nfo0'}).text)
+        except:
+            pass
+
+
+config = [{
+    'name': 'nzbindex',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'nzb_providers',
+            'name': 'nzbindex',
+            'description': 'Free provider, less accurate. See <a href="https://www.nzbindex.com/">NZBIndex</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': True,
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/nzb/omgwtfnzbs.py

@@ -1,24 +1,26 @@
+from urlparse import urlparse, parse_qs
+import time
+
 from couchpotato.core.event import fireEvent
 from couchpotato.core.helpers.encoding import toUnicode, tryUrlencode
 from couchpotato.core.helpers.rss import RSS
 from couchpotato.core.helpers.variable import tryInt
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.nzb.base import NZBProvider
+from couchpotato.core.media._base.providers.nzb.base import NZBProvider
 from dateutil.parser import parse
-from urlparse import urlparse, parse_qs
-import time
+
 
 log = CPLog(__name__)
 
 
-class OMGWTFNZBs(NZBProvider, RSS):
+class Base(NZBProvider, RSS):
 
     urls = {
         'search': 'https://rss.omgwtfnzbs.org/rss-search.php?%s',
         'detail_url': 'https://omgwtfnzbs.org/details.php?id=%s',
     }
 
-    http_time_between_calls = 1  #seconds
+    http_time_between_calls = 1   # Seconds
 
     cat_ids = [
         ([15], ['dvdrip']),
@@ -33,14 +35,14 @@ class OMGWTFNZBs(NZBProvider, RSS):
         if quality['identifier'] in fireEvent('quality.pre_releases', single = True):
             return []
 
-        return super(OMGWTFNZBs, self).search(movie, quality)
+        return super(Base, self).search(movie, quality)
 
     def _searchOnTitle(self, title, movie, quality, results):
 
-        q = '%s %s' % (title, movie['library']['year'])
+        q = '%s %s' % (title, movie['info']['year'])
         params = tryUrlencode({
             'search': q,
-            'catid': ','.join([str(x) for x in self.getCatId(quality['identifier'])]),
+            'catid': ','.join([str(x) for x in self.getCatId(quality)]),
             'user': self.conf('username', default = ''),
             'api': self.conf('api_key', default = ''),
         })
@@ -61,3 +63,40 @@ class OMGWTFNZBs(NZBProvider, RSS):
                 'detail_url': self.urls['detail_url'] % nzb_id,
                 'description': self.getTextElement(nzb, 'description')
             })
+
+
+config = [{
+    'name': 'omgwtfnzbs',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'nzb_providers',
+            'name': 'OMGWTFNZBs',
+            'description': 'See <a href="http://omgwtfnzbs.org/">OMGWTFNZBs</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'api_key',
+                    'label': 'Api Key',
+                    'default': '',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'default': 20,
+                    'type': 'int',
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/awesomehd.py

@@ -1,14 +1,16 @@
-from bs4 import BeautifulSoup
-from couchpotato.core.helpers.variable import tryInt
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentProvider
 import re
 import traceback
 
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.variable import tryInt, getIdentifier
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
+
 log = CPLog(__name__)
 
 
-class AwesomeHD(TorrentProvider):
+class Base(TorrentProvider):
 
     urls = {
         'test': 'https://awesome-hd.net/',
@@ -20,7 +22,7 @@ class AwesomeHD(TorrentProvider):
 
     def _search(self, movie, quality, results):
 
-        data = self.getHTMLData(self.urls['search'] % (self.conf('passkey'), movie['library']['identifier'], self.conf('only_internal')))
+        data = self.getHTMLData(self.urls['search'] % (self.conf('passkey'), getIdentifier(movie), self.conf('only_internal')))
 
         if data:
             try:
@@ -67,3 +69,72 @@ class AwesomeHD(TorrentProvider):
 
             except:
                 log.error('Failed getting results from %s: %s', (self.getName(), traceback.format_exc()))
+
+
+config = [{
+    'name': 'awesomehd',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'Awesome-HD',
+            'description': 'See <a href="https://awesome-hd.net">AHD</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'passkey',
+                    'default': '',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'only_internal',
+                    'advanced': True,
+                    'type': 'bool',
+                    'default': 1,
+                    'description': 'Only search for internal releases.'
+                },
+                {
+                    'name': 'prefer_internal',
+                    'advanced': True,
+                    'type': 'bool',
+                    'default': 1,
+                    'description': 'Favors internal releases over non-internal releases.'
+                },
+                {
+                    'name': 'favor',
+                    'advanced': True,
+                    'default': 'both',
+                    'type': 'dropdown',
+                    'values': [('Encodes & Remuxes', 'both'), ('Encodes', 'encode'), ('Remuxes', 'remux'), ('None', 'none')],
+                    'description': 'Give extra scoring to encodes or remuxes.'
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'type': 'int',
+                    'default': 20,
+                    'description': 'Starting score for each release found via this provider.',
+                },
+            ],
+        },
+    ],
+}]
+

couchpotato/core/media/_base/providers/torrent/base.py

@@ -1,9 +1,11 @@
+import time
 import traceback
+
 from couchpotato.core.helpers.variable import getImdb, md5, cleanHost
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.base import YarrProvider
+from couchpotato.core.media._base.providers.base import YarrProvider
 from couchpotato.environment import Env
-import time
+
 
 log = CPLog(__name__)
 
@@ -15,6 +17,22 @@ class TorrentProvider(YarrProvider):
     proxy_domain = None
     proxy_list = []
 
+    def imdbMatch(self, url, imdbId):
+        if getImdb(url) == imdbId:
+            return True
+
+        if url[:4] == 'http':
+            try:
+                cache_key = md5(url)
+                data = self.getCache(cache_key, url)
+            except IOError:
+                log.error('Failed to open %s.', url)
+                return False
+
+            return getImdb(data) == imdbId
+
+        return False
+
     def getDomain(self, url = ''):
 
         forced_domain = self.conf('domain')

couchpotato/core/media/_base/providers/torrent/bithdtv.py

@@ -1,14 +1,16 @@
+import traceback
+
 from bs4 import BeautifulSoup
-from couchpotato.core.helpers.encoding import tryUrlencode, toUnicode
+from couchpotato.core.helpers.encoding import toUnicode
 from couchpotato.core.helpers.variable import tryInt
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentProvider
-import traceback
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
 
 log = CPLog(__name__)
 
 
-class BiTHDTV(TorrentProvider):
+class Base(TorrentProvider):
 
     urls = {
         'test': 'http://www.bit-hdtv.com/',
@@ -19,18 +21,13 @@ class BiTHDTV(TorrentProvider):
     }
 
     # Searches for movies only - BiT-HDTV's subcategory and resolution search filters appear to be broken
-    cat_id_movies = 7
+    http_time_between_calls = 1  # Seconds
 
-    http_time_between_calls = 1 #seconds
+    def _search(self, media, quality, results):
 
-    def _searchOnTitle(self, title, movie, quality, results):
+        query = self.buildUrl(media)
 
-        arguments = tryUrlencode({
-            'search': '%s %s' % (title.replace(':', ''), movie['library']['year']),
-            'cat': self.cat_id_movies
-        })
-
-        url = "%s&%s" % (self.urls['search'], arguments)
+        url = "%s&%s" % (self.urls['search'], query)
 
         data = self.getHTMLData(url)
 
@@ -43,7 +40,7 @@ class BiTHDTV(TorrentProvider):
             html = BeautifulSoup(data)
 
             try:
-                result_table = html.find('table', attrs = {'width' : '750', 'class' : ''})
+                result_table = html.find('table', attrs = {'width': '750', 'class': ''})
                 if result_table is None:
                     return
 
@@ -77,7 +74,7 @@ class BiTHDTV(TorrentProvider):
     def getMoreInfo(self, item):
         full_description = self.getCache('bithdtv.%s' % item['id'], item['detail_url'], cache_timeout = 25920000)
         html = BeautifulSoup(full_description)
-        nfo_pre = html.find('table', attrs = {'class':'detail'})
+        nfo_pre = html.find('table', attrs = {'class': 'detail'})
         description = toUnicode(nfo_pre.text) if nfo_pre else ''
 
         item['description'] = description
@@ -87,3 +84,55 @@ class BiTHDTV(TorrentProvider):
         return 'logout.php' in output.lower()
 
     loginCheckSuccess = loginSuccess
+
+
+config = [{
+    'name': 'bithdtv',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'BiT-HDTV',
+            'description': 'See <a href="http://bit-hdtv.com">BiT-HDTV</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 20,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/bitsoup.py

@@ -1,14 +1,16 @@
+import traceback
+
 from bs4 import BeautifulSoup
 from couchpotato.core.helpers.encoding import simplifyString, tryUrlencode
 from couchpotato.core.helpers.variable import tryInt
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentProvider
-import traceback
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
 
 log = CPLog(__name__)
 
 
-class Bitsoup(TorrentProvider):
+class Base(TorrentProvider):
 
     urls = {
         'test': 'https://www.bitsoup.me/',
@@ -18,16 +20,17 @@ class Bitsoup(TorrentProvider):
         'baseurl': 'https://www.bitsoup.me/%s',
     }
 
-    http_time_between_calls = 1 #seconds
+    http_time_between_calls = 1  # Seconds
 
     def _searchOnTitle(self, title, movie, quality, results):
 
-        q = '"%s" %s' % (simplifyString(title), movie['library']['year'])
+        q = '"%s" %s' % (simplifyString(title), movie['info']['year'])
         arguments = tryUrlencode({
             'search': q,
         })
         url = "%s&%s" % (self.urls['search'], arguments)
 
+        url = self.urls['search'] % self.buildUrl(movie, quality)
         data = self.getHTMLData(url)
 
         if data:
@@ -71,7 +74,6 @@ class Bitsoup(TorrentProvider):
             except:
                 log.error('Failed getting results from %s: %s', (self.getName(), traceback.format_exc()))
 
-
     def getLoginParams(self):
         return {
             'username': self.conf('username'),
@@ -79,9 +81,59 @@ class Bitsoup(TorrentProvider):
             'ssl': 'yes',
         }
 
-
     def loginSuccess(self, output):
         return 'logout.php' in output.lower()
 
     loginCheckSuccess = loginSuccess
 
+
+config = [{
+    'name': 'bitsoup',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'Bitsoup',
+            'description': 'See <a href="https://bitsoup.me">Bitsoup</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 20,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/hdbits.py

@@ -0,0 +1,114 @@
+import re
+import json
+import traceback
+
+from couchpotato.core.helpers.variable import tryInt, getIdentifier
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
+
+log = CPLog(__name__)
+
+
+class Base(TorrentProvider):
+
+    urls = {
+        'test': 'https://hdbits.org/',
+        'detail': 'https://hdbits.org/details.php?id=%s',
+        'download': 'https://hdbits.org/download.php?id=%s&passkey=%s',
+        'api': 'https://hdbits.org/api/torrents'
+    }
+
+    http_time_between_calls = 1  # Seconds
+
+    def _post_query(self, **params):
+
+        post_data = {
+            'username': self.conf('username'),
+            'passkey': self.conf('passkey')
+        }
+        post_data.update(params)
+
+        try:
+            result = self.getJsonData(self.urls['api'], data = json.dumps(post_data))
+
+            if result:
+                if result['status'] != 0:
+                    log.error('Error searching hdbits: %s' % result['message'])
+                else:
+                    return result['data']
+        except:
+            pass
+
+        return None
+
+    def _search(self, movie, quality, results):
+
+        match = re.match(r'tt(\d{7})', getIdentifier(movie))
+
+        data = self._post_query(imdb = {'id': match.group(1)})
+
+        if data:
+            try:
+                for result in data:
+                    results.append({
+                        'id': result['id'],
+                        'name': result['name'],
+                        'url': self.urls['download'] % (result['id'], self.conf('passkey')),
+                        'detail_url': self.urls['detail'] % result['id'],
+                        'size': tryInt(result['size']) / 1024 / 1024,
+                        'seeders': tryInt(result['seeders']),
+                        'leechers': tryInt(result['leechers'])
+                    })
+            except:
+                log.error('Failed getting results from %s: %s', (self.getName(), traceback.format_exc()))
+
+
+config = [{
+    'name': 'hdbits',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'HDBits',
+            'description': 'See <a href="http://hdbits.org">HDBits</a>',
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'passkey',
+                    'default': '',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                },
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/ilovetorrents.py

@@ -1,15 +1,17 @@
+import re
+import traceback
+
 from bs4 import BeautifulSoup
 from couchpotato.core.helpers.encoding import toUnicode, tryUrlencode
 from couchpotato.core.helpers.variable import tryInt
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentProvider
-import re
-import traceback
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
 
 log = CPLog(__name__)
 
 
-class ILoveTorrents(TorrentProvider):
+class Base(TorrentProvider):
 
     urls = {
         'download': 'https://www.ilovetorrents.me/%s',
@@ -21,9 +23,9 @@ class ILoveTorrents(TorrentProvider):
     }
 
     cat_ids = [
-       (['41'], ['720p', '1080p', 'brrip']),
-       (['19'], ['cam', 'ts', 'dvdrip', 'tc', 'r5', 'scr']),
-       (['20'], ['dvdr'])
+        (['41'], ['720p', '1080p', 'brrip']),
+        (['19'], ['cam', 'ts', 'dvdrip', 'tc', 'r5', 'scr']),
+        (['20'], ['dvdr'])
     ]
 
     cat_backup_id = 200
@@ -34,11 +36,11 @@ class ILoveTorrents(TorrentProvider):
 
         page = 0
         total_pages = 1
-        cats = self.getCatId(quality['identifier'])
+        cats = self.getCatId(quality)
 
         while page < total_pages:
 
-            movieTitle = tryUrlencode('"%s" %s' % (title, movie['library']['year']))
+            movieTitle = tryUrlencode('"%s" %s' % (title, movie['info']['year']))
             search_url = self.urls['search'] % (movieTitle, page, cats[0])
             page += 1
 
@@ -77,7 +79,7 @@ class ILoveTorrents(TorrentProvider):
                                 return confirmed + trusted + vip + moderated
 
                             id = re.search('id=(?P<id>\d+)&', link).group('id')
-                            url = self.urls['download'] % (download)
+                            url = self.urls['download'] % download
 
                             fileSize = self.parseSize(result.select('td.rowhead')[5].text)
                             results.append({
@@ -111,7 +113,7 @@ class ILoveTorrents(TorrentProvider):
             try:
                 full_description = self.getHTMLData(item['detail_url'])
                 html = BeautifulSoup(full_description)
-                nfo_pre = html.find('td', attrs = {'class':'main'}).findAll('table')[1]
+                nfo_pre = html.find('td', attrs = {'class': 'main'}).findAll('table')[1]
                 description = toUnicode(nfo_pre.text) if nfo_pre else ''
             except:
                 log.error('Failed getting more info for %s', item['name'])
@@ -126,3 +128,60 @@ class ILoveTorrents(TorrentProvider):
         return 'logout.php' in output.lower()
 
     loginCheckSuccess = loginSuccess
+
+
+config = [{
+    'name': 'ilovetorrents',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'ILoveTorrents',
+            'description': 'Where the Love of Torrents is Born',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False
+                },
+                {
+                    'name': 'username',
+                    'label': 'Username',
+                    'type': 'string',
+                    'default': '',
+                    'description': 'The user name for your ILT account',
+                },
+                {
+                    'name': 'password',
+                    'label': 'Password',
+                    'type': 'password',
+                    'default': '',
+                    'description': 'The password for your ILT account.',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        }
+    ]
+}]

couchpotato/core/media/_base/providers/torrent/iptorrents.py

@@ -0,0 +1,171 @@
+import traceback
+
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.encoding import tryUrlencode
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+import six
+
+
+log = CPLog(__name__)
+
+
+class Base(TorrentProvider):
+
+    urls = {
+        'test': 'https://www.iptorrents.com/',
+        'base_url': 'https://www.iptorrents.com',
+        'login': 'https://www.iptorrents.com/torrents/',
+        'login_check': 'https://www.iptorrents.com/inbox.php',
+        'search': 'https://www.iptorrents.com/torrents/?%s%%s&q=%s&qf=ti&p=%%d',
+    }
+
+    http_time_between_calls = 1  # Seconds
+    cat_backup_id = None
+
+    def buildUrl(self, title, media, quality):
+        return self._buildUrl(title.replace(':', ''), quality)
+
+    def _buildUrl(self, query, quality):
+
+        cat_ids = self.getCatId(quality)
+
+        if not cat_ids:
+            log.warning('Unable to find category ids for identifier "%s"', quality.get('identifier'))
+            return None
+
+        return self.urls['search'] % ("&".join(("l%d=" % x) for x in cat_ids), tryUrlencode(query).replace('%', '%%'))
+
+    def _searchOnTitle(self, title, media, quality, results):
+
+        freeleech = '' if not self.conf('freeleech') else '&free=on'
+
+        base_url = self.buildUrl(title, media, quality)
+        if not base_url: return
+
+        pages = 1
+        current_page = 1
+        while current_page <= pages and not self.shuttingDown():
+            data = self.getHTMLData(base_url % (freeleech, current_page))
+
+            if data:
+                html = BeautifulSoup(data)
+
+                try:
+                    page_nav = html.find('span', attrs = {'class': 'page_nav'})
+                    if page_nav:
+                        next_link = page_nav.find("a", text = "Next")
+                        if next_link:
+                            final_page_link = next_link.previous_sibling.previous_sibling
+                            pages = int(final_page_link.string)
+
+                    result_table = html.find('table', attrs = {'class': 'torrents'})
+
+                    if not result_table or 'nothing found!' in data.lower():
+                        return
+
+                    entries = result_table.find_all('tr')
+
+                    for result in entries[1:]:
+
+                        torrent = result.find_all('td')
+                        if len(torrent) <= 1:
+                            break
+
+                        torrent = torrent[1].find('a')
+
+                        torrent_id = torrent['href'].replace('/details.php?id=', '')
+                        torrent_name = six.text_type(torrent.string)
+                        torrent_download_url = self.urls['base_url'] + (result.find_all('td')[3].find('a'))['href'].replace(' ', '.')
+                        torrent_details_url = self.urls['base_url'] + torrent['href']
+                        torrent_size = self.parseSize(result.find_all('td')[5].string)
+                        torrent_seeders = tryInt(result.find('td', attrs = {'class': 'ac t_seeders'}).string)
+                        torrent_leechers = tryInt(result.find('td', attrs = {'class': 'ac t_leechers'}).string)
+
+                        results.append({
+                            'id': torrent_id,
+                            'name': torrent_name,
+                            'url': torrent_download_url,
+                            'detail_url': torrent_details_url,
+                            'size': torrent_size,
+                            'seeders': torrent_seeders,
+                            'leechers': torrent_leechers,
+                        })
+
+                except:
+                    log.error('Failed to parsing %s: %s', (self.getName(), traceback.format_exc()))
+                    break
+
+            current_page += 1
+
+    def getLoginParams(self):
+        return {
+            'username': self.conf('username'),
+            'password': self.conf('password'),
+            'login': 'submit',
+        }
+
+    def loginSuccess(self, output):
+        return 'don\'t have an account' not in output.lower()
+
+    def loginCheckSuccess(self, output):
+        return '/logout.php' in output.lower()
+
+
+config = [{
+    'name': 'iptorrents',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'IPTorrents',
+            'description': 'See <a href="http://www.iptorrents.com">IPTorrents</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'freeleech',
+                    'default': 0,
+                    'type': 'bool',
+                    'description': 'Only search for [FreeLeech] torrents.',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/kickasstorrents.py

@@ -1,14 +1,16 @@
-from bs4 import BeautifulSoup
-from couchpotato.core.helpers.variable import tryInt
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentMagnetProvider
 import re
 import traceback
 
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.variable import tryInt, getIdentifier
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentMagnetProvider
+
+
 log = CPLog(__name__)
 
 
-class KickAssTorrents(TorrentMagnetProvider):
+class Base(TorrentMagnetProvider):
 
     urls = {
         'detail': '%s/%s',
@@ -24,7 +26,7 @@ class KickAssTorrents(TorrentMagnetProvider):
         (['dvd'], ['dvdr']),
     ]
 
-    http_time_between_calls = 1  #seconds
+    http_time_between_calls = 1  # Seconds
     cat_backup_id = None
 
     proxy_list = [
@@ -34,13 +36,13 @@ class KickAssTorrents(TorrentMagnetProvider):
         'http://www.kickassproxy.info',
     ]
 
-    def _search(self, movie, quality, results):
+    def _search(self, media, quality, results):
 
-        data = self.getHTMLData(self.urls['search'] % (self.getDomain(), 'm', movie['library']['identifier'].replace('tt', '')))
+        data = self.getHTMLData(self.urls['search'] % (self.getDomain(), 'm', getIdentifier(media).replace('tt', '')))
 
         if data:
 
-            cat_ids = self.getCatId(quality['identifier'])
+            cat_ids = self.getCatId(quality)
             table_order = ['name', 'size', None, 'age', 'seeds', 'leechers']
 
             try:
@@ -108,7 +110,56 @@ class KickAssTorrents(TorrentMagnetProvider):
         return tryInt(age)
 
     def isEnabled(self):
-        return super(KickAssTorrents, self).isEnabled() and self.getDomain()
+        return super(Base, self).isEnabled() and self.getDomain()
 
     def correctProxy(self, data):
         return 'search query' in data.lower()
+
+
+config = [{
+    'name': 'kickasstorrents',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'KickAssTorrents',
+            'description': 'See <a href="https://kat.ph/">KickAssTorrents</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': True,
+                },
+                {
+                    'name': 'domain',
+                    'advanced': True,
+                    'label': 'Proxy server',
+                    'description': 'Domain for requests, keep empty to let CouchPotato pick.',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/passthepopcorn.py

@@ -1,19 +1,21 @@
-from couchpotato.core.helpers.encoding import tryUrlencode
-from couchpotato.core.helpers.variable import getTitle, tryInt, mergeDicts
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentProvider
-from dateutil.parser import parse
 import htmlentitydefs
 import json
 import re
 import time
 import traceback
+
+from couchpotato.core.helpers.encoding import tryUrlencode
+from couchpotato.core.helpers.variable import getTitle, tryInt, mergeDicts, getIdentifier
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+from dateutil.parser import parse
 import six
 
+
 log = CPLog(__name__)
 
 
-class PassThePopcorn(TorrentProvider):
+class Base(TorrentProvider):
 
     urls = {
         'domain': 'https://tls.passthepopcorn.me',
@@ -26,43 +28,15 @@ class PassThePopcorn(TorrentProvider):
 
     http_time_between_calls = 2
 
-    quality_search_params = {
-        'bd50':     {'media': 'Blu-ray', 'format': 'BD50'},
-        '1080p':    {'resolution': '1080p'},
-        '720p':     {'resolution': '720p'},
-        'brrip':    {'media': 'Blu-ray'},
-        'dvdr':     {'resolution': 'anysd'},
-        'dvdrip':   {'media': 'DVD'},
-        'scr':      {'media': 'DVD-Screener'},
-        'r5':       {'media': 'R5'},
-        'tc':       {'media': 'TC'},
-        'ts':       {'media': 'TS'},
-        'cam':      {'media': 'CAM'}
-    }
+    def _search(self, media, quality, results):
 
-    post_search_filters = {
-        'bd50':     {'Codec': ['BD50']},
-        '1080p':    {'Resolution': ['1080p']},
-        '720p':     {'Resolution': ['720p']},
-        'brrip':    {'Source': ['Blu-ray'], 'Quality': ['High Definition'], 'Container': ['!ISO']},
-        'dvdr':     {'Codec': ['DVD5', 'DVD9']},
-        'dvdrip':   {'Source': ['DVD'], 'Codec': ['!DVD5', '!DVD9']},
-        'scr':      {'Source': ['DVD-Screener']},
-        'r5':       {'Source': ['R5']},
-        'tc':       {'Source': ['TC']},
-        'ts':       {'Source': ['TS']},
-        'cam':      {'Source': ['CAM']}
-    }
-
-    def _search(self, movie, quality, results):
-
-        movie_title = getTitle(movie['library'])
+        movie_title = getTitle(media)
         quality_id = quality['identifier']
 
         params = mergeDicts(self.quality_search_params[quality_id].copy(), {
             'order_by': 'relevance',
             'order_way': 'descending',
-            'searchstr': movie['library']['identifier']
+            'searchstr': getIdentifier(media)
         })
 
         url = '%s?json=noredirect&%s' % (self.urls['torrent'], tryUrlencode(params))
@@ -162,23 +136,23 @@ class PassThePopcorn(TorrentProvider):
 
     def htmlToUnicode(self, text):
         def fixup(m):
-            text = m.group(0)
-            if text[:2] == "&#":
+            txt = m.group(0)
+            if txt[:2] == "&#":
                 # character reference
                 try:
-                    if text[:3] == "&#x":
-                        return unichr(int(text[3:-1], 16))
+                    if txt[:3] == "&#x":
+                        return unichr(int(txt[3:-1], 16))
                     else:
-                        return unichr(int(text[2:-1]))
+                        return unichr(int(txt[2:-1]))
                 except ValueError:
                     pass
             else:
                 # named entity
                 try:
-                    text = unichr(htmlentitydefs.name2codepoint[text[1:-1]])
+                    txt = unichr(htmlentitydefs.name2codepoint[txt[1:-1]])
                 except KeyError:
                     pass
-            return text # leave as is
+            return txt  # leave as is
         return re.sub("&#?\w+;", fixup, six.u('%s') % text)
 
     def unicodeToASCII(self, text):
@@ -190,11 +164,11 @@ class PassThePopcorn(TorrentProvider):
 
     def getLoginParams(self):
         return {
-             'username': self.conf('username'),
-             'password': self.conf('password'),
-             'passkey': self.conf('passkey'),
-             'keeplogged': '1',
-             'login': 'Login'
+            'username': self.conf('username'),
+            'password': self.conf('password'),
+            'passkey': self.conf('passkey'),
+            'keeplogged': '1',
+            'login': 'Login'
         }
 
     def loginSuccess(self, output):
@@ -204,3 +178,89 @@ class PassThePopcorn(TorrentProvider):
             return False
 
     loginCheckSuccess = loginSuccess
+
+
+config = [{
+    'name': 'passthepopcorn',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'PassThePopcorn',
+            'description': 'See <a href="https://passthepopcorn.me">PassThePopcorn.me</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False
+                },
+                {
+                    'name': 'domain',
+                    'advanced': True,
+                    'label': 'Proxy server',
+                    'description': 'Domain for requests (HTTPS only!), keep empty to use default (tls.passthepopcorn.me).',
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'passkey',
+                    'default': '',
+                },
+                {
+                    'name': 'prefer_golden',
+                    'advanced': True,
+                    'type': 'bool',
+                    'label': 'Prefer golden',
+                    'default': 1,
+                    'description': 'Favors Golden Popcorn-releases over all other releases.'
+                },
+                {
+                    'name': 'prefer_scene',
+                    'advanced': True,
+                    'type': 'bool',
+                    'label': 'Prefer scene',
+                    'default': 0,
+                    'description': 'Favors scene-releases over non-scene releases.'
+                },
+                {
+                    'name': 'require_approval',
+                    'advanced': True,
+                    'type': 'bool',
+                    'label': 'Require approval',
+                    'default': 0,
+                    'description': 'Require staff-approval for releases to be accepted.'
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 20,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        }
+    ]
+}]

couchpotato/core/media/_base/providers/torrent/publichd.py

@@ -1,16 +1,19 @@
-from bs4 import BeautifulSoup
-from couchpotato.core.helpers.encoding import tryUrlencode, toUnicode
-from couchpotato.core.helpers.variable import tryInt
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentMagnetProvider
 from urlparse import parse_qs
 import re
 import traceback
 
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.encoding import tryUrlencode, toUnicode
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentMagnetProvider
+import six
+
+
 log = CPLog(__name__)
 
 
-class PublicHD(TorrentMagnetProvider):
+class Base(TorrentMagnetProvider):
 
     urls = {
         'test': 'https://publichd.se',
@@ -24,13 +27,15 @@ class PublicHD(TorrentMagnetProvider):
         if not quality.get('hd', False):
             return []
 
-        return super(PublicHD, self).search(movie, quality)
+        return super(Base, self).search(movie, quality)
 
-    def _searchOnTitle(self, title, movie, quality, results):
+    def _search(self, media, quality, results):
+
+        query = self.buildUrl(media)
 
         params = tryUrlencode({
-            'page':'torrents',
-            'search': '%s %s' % (title, movie['library']['year']),
+            'page': 'torrents',
+            'search': query,
             'active': 1,
         })
 
@@ -54,7 +59,7 @@ class PublicHD(TorrentMagnetProvider):
 
                         results.append({
                             'id': url['id'][0],
-                            'name': info_url.string,
+                            'name': six.text_type(info_url.string),
                             'url': download['href'],
                             'detail_url': self.urls['detail'] % url['id'][0],
                             'size': self.parseSize(result.find_all('td')[7].string),
@@ -86,3 +91,46 @@ class PublicHD(TorrentMagnetProvider):
 
         item['description'] = description
         return item
+
+
+config = [{
+    'name': 'publichd',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'PublicHD',
+            'description': 'Public Torrent site with only HD content. See <a href="https://publichd.se/">PublicHD</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': True,
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/sceneaccess.py

@@ -0,0 +1,134 @@
+import traceback
+
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.encoding import toUnicode
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
+
+log = CPLog(__name__)
+
+
+class Base(TorrentProvider):
+
+    urls = {
+        'test': 'https://www.sceneaccess.eu/',
+        'login': 'https://www.sceneaccess.eu/login',
+        'login_check': 'https://www.sceneaccess.eu/inbox',
+        'detail': 'https://www.sceneaccess.eu/details?id=%s',
+        'search': 'https://www.sceneaccess.eu/browse?c%d=%d',
+        'archive': 'https://www.sceneaccess.eu/archive?&c%d=%d',
+        'download': 'https://www.sceneaccess.eu/%s',
+    }
+
+    http_time_between_calls = 1  # Seconds
+
+    def _search(self, media, quality, results):
+
+        url = self.buildUrl(media, quality)
+        data = self.getHTMLData(url)
+
+        if data:
+            html = BeautifulSoup(data)
+
+            try:
+                resultsTable = html.find('table', attrs = {'id': 'torrents-table'})
+                if resultsTable is None:
+                    return
+
+                entries = resultsTable.find_all('tr', attrs = {'class': 'tt_row'})
+                for result in entries:
+
+                    link = result.find('td', attrs = {'class': 'ttr_name'}).find('a')
+                    url = result.find('td', attrs = {'class': 'td_dl'}).find('a')
+                    leechers = result.find('td', attrs = {'class': 'ttr_leechers'}).find('a')
+                    torrent_id = link['href'].replace('details?id=', '')
+
+                    results.append({
+                        'id': torrent_id,
+                        'name': link['title'],
+                        'url': self.urls['download'] % url['href'],
+                        'detail_url': self.urls['detail'] % torrent_id,
+                        'size': self.parseSize(result.find('td', attrs = {'class': 'ttr_size'}).contents[0]),
+                        'seeders': tryInt(result.find('td', attrs = {'class': 'ttr_seeders'}).find('a').string),
+                        'leechers': tryInt(leechers.string) if leechers else 0,
+                        'get_more_info': self.getMoreInfo,
+                    })
+
+            except:
+                log.error('Failed getting results from %s: %s', (self.getName(), traceback.format_exc()))
+
+    def getMoreInfo(self, item):
+        full_description = self.getCache('sceneaccess.%s' % item['id'], item['detail_url'], cache_timeout = 25920000)
+        html = BeautifulSoup(full_description)
+        nfo_pre = html.find('div', attrs = {'id': 'details_table'})
+        description = toUnicode(nfo_pre.text) if nfo_pre else ''
+
+        item['description'] = description
+        return item
+
+    # Login
+    def getLoginParams(self):
+        return {
+            'username': self.conf('username'),
+            'password': self.conf('password'),
+            'submit': 'come on in',
+        }
+
+    def loginSuccess(self, output):
+        return '/inbox' in output.lower()
+
+    loginCheckSuccess = loginSuccess
+
+
+config = [{
+    'name': 'sceneaccess',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'SceneAccess',
+            'description': 'See <a href="https://sceneaccess.eu/">SceneAccess</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 20,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/thepiratebay.py

@@ -1,29 +1,24 @@
-from bs4 import BeautifulSoup
-from couchpotato.core.helpers.encoding import toUnicode, tryUrlencode
-from couchpotato.core.helpers.variable import tryInt
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentMagnetProvider
 import re
 import traceback
+
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.encoding import toUnicode
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentMagnetProvider
 import six
 
+
 log = CPLog(__name__)
 
 
-class ThePirateBay(TorrentMagnetProvider):
+class Base(TorrentMagnetProvider):
 
     urls = {
-        'detail': '%s/torrent/%s',
-        'search': '%s/search/%s/%s/7/%s'
+         'detail': '%s/torrent/%s',
+         'search': '%s/search/%%s/%%s/7/%%s'
     }
 
-    cat_ids = [
-        ([207], ['720p', '1080p']),
-        ([201], ['cam', 'ts', 'dvdrip', 'tc', 'r5', 'scr']),
-        ([201, 207], ['brrip']),
-        ([202], ['dvdr'])
-    ]
-
     cat_backup_id = 200
     disable_provider = False
     http_time_between_calls = 0
@@ -34,21 +29,24 @@ class ThePirateBay(TorrentMagnetProvider):
         'http://pirateproxy.ca',
         'http://tpb.al',
         'http://www.tpb.gr',
-        'http://nl.tpb.li',
+        'http://bayproxy.me',
         'http://proxybay.eu',
-        'https://www.getpirate.com',
-        'http://piratebay.io', 
+        'http://www.getpirate.com',
+        'http://piratebay.io',
     ]
 
-    def _searchOnTitle(self, title, movie, quality, results):
+    def _search(self, media, quality, results):
 
         page = 0
         total_pages = 1
-        cats = self.getCatId(quality['identifier'])
+        cats = self.getCatId(quality)
+
+        base_search_url = self.urls['search'] % self.getDomain()
 
         while page < total_pages:
 
-            search_url = self.urls['search'] % (self.getDomain(), tryUrlencode('"%s" %s' % (title, movie['library']['year'])), page, ','.join(str(x) for x in cats))
+            search_url = base_search_url % self.buildUrl(media, page, cats)
+
             page += 1
 
             data = self.getHTMLData(search_url)
@@ -88,7 +86,7 @@ class ThePirateBay(TorrentMagnetProvider):
 
                             results.append({
                                 'id': re.search('/(?P<id>\d+)/', link['href']).group('id'),
-                                'name': link.string,
+                                'name': six.text_type(link.string),
                                 'url': download['href'],
                                 'detail_url': self.getDomain(link['href']),
                                 'size': self.parseSize(size),
@@ -102,7 +100,7 @@ class ThePirateBay(TorrentMagnetProvider):
                     log.error('Failed getting results from %s: %s', (self.getName(), traceback.format_exc()))
 
     def isEnabled(self):
-        return super(ThePirateBay, self).isEnabled() and self.getDomain()
+        return super(Base, self).isEnabled() and self.getDomain()
 
     def correctProxy(self, data):
         return 'title="Pirate Search"' in data
@@ -115,3 +113,52 @@ class ThePirateBay(TorrentMagnetProvider):
 
         item['description'] = description
         return item
+
+
+config = [{
+    'name': 'thepiratebay',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'ThePirateBay',
+            'description': 'The world\'s largest bittorrent tracker. See <a href="http://fucktimkuik.org/">ThePirateBay</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False
+                },
+                {
+                    'name': 'domain',
+                    'advanced': True,
+                    'label': 'Proxy server',
+                    'description': 'Domain for requests, keep empty to let CouchPotato pick.',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        }
+    ]
+}]

couchpotato/core/media/_base/providers/torrent/torrentbytes.py

@@ -1,14 +1,16 @@
+import traceback
+
 from bs4 import BeautifulSoup
 from couchpotato.core.helpers.encoding import tryUrlencode
 from couchpotato.core.helpers.variable import tryInt
 from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.torrent.base import TorrentProvider
-import traceback
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
 
 log = CPLog(__name__)
 
 
-class TorrentBytes(TorrentProvider):
+class Base(TorrentProvider):
 
     urls = {
         'test': 'https://www.torrentbytes.net/',
@@ -29,19 +31,19 @@ class TorrentBytes(TorrentProvider):
         ([20], ['dvdr']),
     ]
 
-    http_time_between_calls = 1 #seconds
+    http_time_between_calls = 1  # Seconds
     cat_backup_id = None
 
     def _searchOnTitle(self, title, movie, quality, results):
 
-        url = self.urls['search'] % (tryUrlencode('%s %s' % (title.replace(':', ''), movie['library']['year'])), self.getCatId(quality['identifier'])[0])
+        url = self.urls['search'] % (tryUrlencode('%s %s' % (title.replace(':', ''), movie['info']['year'])), self.getCatId(quality)[0])
         data = self.getHTMLData(url)
 
         if data:
             html = BeautifulSoup(data)
 
             try:
-                result_table = html.find('table', attrs = {'border' : '1'})
+                result_table = html.find('table', attrs = {'border': '1'})
                 if not result_table:
                     return
 
@@ -50,7 +52,7 @@ class TorrentBytes(TorrentProvider):
                 for result in entries[1:]:
                     cells = result.find_all('td')
 
-                    link = cells[1].find('a', attrs = {'class' : 'index'})
+                    link = cells[1].find('a', attrs = {'class': 'index'})
 
                     full_id = link['href'].replace('details.php?id=', '')
                     torrent_id = full_id[:6]
@@ -80,3 +82,54 @@ class TorrentBytes(TorrentProvider):
 
     loginCheckSuccess = loginSuccess
 
+
+config = [{
+    'name': 'torrentbytes',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'TorrentBytes',
+            'description': 'See <a href="http://torrentbytes.net">TorrentBytes</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 20,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/torrentday.py

@@ -0,0 +1,113 @@
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
+log = CPLog(__name__)
+
+
+class Base(TorrentProvider):
+
+    urls = {
+        'test': 'http://www.td.af/',
+        'login': 'http://www.td.af/torrents/',
+        'login_check': 'http://www.torrentday.com/userdetails.php',
+        'detail': 'http://www.td.af/details.php?id=%s',
+        'search': 'http://www.td.af/V3/API/API.php',
+        'download': 'http://www.td.af/download.php/%s/%s',
+    }
+
+    http_time_between_calls = 1  # Seconds
+
+    def _search(self, media, quality, results):
+
+        query = self.buildUrl(media)
+
+        data = {
+            '/browse.php?': None,
+            'cata': 'yes',
+            'jxt': 8,
+            'jxw': 'b',
+            'search': query,
+        }
+
+        data = self.getJsonData(self.urls['search'], data = data)
+        try: torrents = data.get('Fs', [])[0].get('Cn', {}).get('torrents', [])
+        except: return
+
+        for torrent in torrents:
+            results.append({
+                'id': torrent['id'],
+                'name': torrent['name'],
+                'url': self.urls['download'] % (torrent['id'], torrent['fname']),
+                'detail_url': self.urls['detail'] % torrent['id'],
+                'size': self.parseSize(torrent.get('size')),
+                'seeders': tryInt(torrent.get('seed')),
+                'leechers': tryInt(torrent.get('leech')),
+            })
+
+    def getLoginParams(self):
+        return {
+            'username': self.conf('username'),
+            'password': self.conf('password'),
+            'submit.x': 18,
+            'submit.y': 11,
+            'submit': 'submit',
+        }
+
+    def loginSuccess(self, output):
+        return 'Password not correct' not in output
+
+    def loginCheckSuccess(self, output):
+        return 'logout.php' in output.lower()
+
+
+config = [{
+    'name': 'torrentday',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'TorrentDay',
+            'description': 'See <a href="http://www.td.af/">TorrentDay</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/torrentleech.py

@@ -0,0 +1,125 @@
+import traceback
+
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+import six
+
+
+log = CPLog(__name__)
+
+
+class Base(TorrentProvider):
+
+    urls = {
+        'test': 'http://www.torrentleech.org/',
+        'login': 'http://www.torrentleech.org/user/account/login/',
+        'login_check': 'http://torrentleech.org/user/messages',
+        'detail': 'http://www.torrentleech.org/torrent/%s',
+        'search': 'http://www.torrentleech.org/torrents/browse/index/query/%s/categories/%d',
+        'download': 'http://www.torrentleech.org%s',
+    }
+
+    http_time_between_calls = 1  # Seconds
+    cat_backup_id = None
+
+    def _search(self, media, quality, results):
+
+        url = self.urls['search'] % self.buildUrl(media, quality)
+
+        data = self.getHTMLData(url)
+
+        if data:
+            html = BeautifulSoup(data)
+
+            try:
+                result_table = html.find('table', attrs = {'id': 'torrenttable'})
+                if not result_table:
+                    return
+
+                entries = result_table.find_all('tr')
+
+                for result in entries[1:]:
+
+                    link = result.find('td', attrs = {'class': 'name'}).find('a')
+                    url = result.find('td', attrs = {'class': 'quickdownload'}).find('a')
+                    details = result.find('td', attrs = {'class': 'name'}).find('a')
+
+                    results.append({
+                        'id': link['href'].replace('/torrent/', ''),
+                        'name': six.text_type(link.string),
+                        'url': self.urls['download'] % url['href'],
+                        'detail_url': self.urls['download'] % details['href'],
+                        'size': self.parseSize(result.find_all('td')[4].string),
+                        'seeders': tryInt(result.find('td', attrs = {'class': 'seeders'}).string),
+                        'leechers': tryInt(result.find('td', attrs = {'class': 'leechers'}).string),
+                    })
+
+            except:
+                log.error('Failed to parsing %s: %s', (self.getName(), traceback.format_exc()))
+
+    def getLoginParams(self):
+        return {
+            'username': self.conf('username'),
+            'password': self.conf('password'),
+            'remember_me': 'on',
+            'login': 'submit',
+        }
+
+    def loginSuccess(self, output):
+        return '/user/account/logout' in output.lower() or 'welcome back' in output.lower()
+
+    loginCheckSuccess = loginSuccess
+
+
+config = [{
+    'name': 'torrentleech',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'TorrentLeech',
+            'description': 'See <a href="http://torrentleech.org">TorrentLeech</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 20,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/torrentpotato.py

@@ -1,45 +1,40 @@
-from couchpotato.core.helpers.encoding import tryUrlencode, toUnicode
-from couchpotato.core.helpers.variable import splitString, tryInt, tryFloat
-from couchpotato.core.logger import CPLog
-from couchpotato.core.providers.base import ResultList
-from couchpotato.core.providers.torrent.base import TorrentProvider
 from urlparse import urlparse
 import re
 import traceback
 
+from couchpotato.core.helpers.encoding import toUnicode
+from couchpotato.core.helpers.variable import splitString, tryInt, tryFloat
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.base import ResultList
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
+
 log = CPLog(__name__)
 
 
-class TorrentPotato(TorrentProvider):
+class Base(TorrentProvider):
 
     urls = {}
     limits_reached = {}
 
     http_time_between_calls = 1  # Seconds
 
-    def search(self, movie, quality):
+    def search(self, media, quality):
         hosts = self.getHosts()
 
-        results = ResultList(self, movie, quality, imdb_results = True)
+        results = ResultList(self, media, quality, imdb_results = True)
 
         for host in hosts:
             if self.isDisabled(host):
                 continue
 
-            self._searchOnHost(host, movie, quality, results)
+            self._searchOnHost(host, media, quality, results)
 
         return results
 
-    def _searchOnHost(self, host, movie, quality, results):
+    def _searchOnHost(self, host, media, quality, results):
 
-        arguments = tryUrlencode({
-            'user': host['name'],
-            'passkey': host['pass_key'],
-            'imdbid': movie['library']['identifier']
-        })
-        url = '%s?%s' % (host['host'], arguments)
-
-        torrents = self.getJsonData(url, cache_timeout = 1800)
+        torrents = self.getJsonData(self.buildUrl(media, host), cache_timeout = 1800)
 
         if torrents:
             try:
@@ -75,7 +70,7 @@ class TorrentPotato(TorrentProvider):
         pass_keys = splitString(self.conf('pass_key'), clean = False)
         extra_score = splitString(self.conf('extra_score'), clean = False)
 
-        list = []
+        host_list = []
         for nr in range(len(hosts)):
 
             try: key = pass_keys[nr]
@@ -93,7 +88,7 @@ class TorrentPotato(TorrentProvider):
             try: seed_time = seed_times[nr]
             except: seed_time = ''
 
-            list.append({
+            host_list.append({
                 'use': uses[nr],
                 'host': host,
                 'name': name,
@@ -103,14 +98,14 @@ class TorrentPotato(TorrentProvider):
                 'extra_score': tryInt(extra_score[nr]) if len(extra_score) > nr else 0
             })
 
-        return list
+        return host_list
 
     def belongsTo(self, url, provider = None, host = None):
 
         hosts = self.getHosts()
 
         for host in hosts:
-            result = super(TorrentPotato, self).belongsTo(url, host = host['host'], provider = provider)
+            result = super(Base, self).belongsTo(url, host = host['host'], provider = provider)
             if result:
                 return result
 
@@ -127,3 +122,66 @@ class TorrentPotato(TorrentProvider):
             return False
 
         return TorrentProvider.isEnabled(self) and host['host'] and host['pass_key'] and int(host['use'])
+
+
+config = [{
+    'name': 'torrentpotato',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'TorrentPotato',
+            'order': 10,
+            'description': 'CouchPotato torrent provider. Checkout <a href="https://github.com/RuudBurger/CouchPotatoServer/wiki/CouchPotato-Torrent-Provider">the wiki page about this provider</a> for more info.',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'use',
+                    'default': ''
+                },
+                {
+                    'name': 'host',
+                    'default': '',
+                    'description': 'The url path of your TorrentPotato provider.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'default': '0',
+                    'description': 'Starting score for each release found via this provider.',
+                },
+                {
+                    'name': 'name',
+                    'label': 'Username',
+                    'default': '',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'default': '1',
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'default': '40',
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'pass_key',
+                    'default': ',',
+                    'label': 'Pass Key',
+                    'description': 'Can be found on your profile page',
+                    'type': 'combined',
+                    'combine': ['use', 'host', 'pass_key', 'name', 'seed_ratio', 'seed_time', 'extra_score'],
+                },
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/torrentshack.py

@@ -0,0 +1,130 @@
+import traceback
+
+from bs4 import BeautifulSoup
+from couchpotato.core.helpers.variable import tryInt
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+import six
+
+
+log = CPLog(__name__)
+
+
+class Base(TorrentProvider):
+
+    urls = {
+        'test': 'https://torrentshack.net/',
+        'login': 'https://torrentshack.net/login.php',
+        'login_check': 'https://torrentshack.net/inbox.php',
+        'detail': 'https://torrentshack.net/torrent/%s',
+        'search': 'https://torrentshack.net/torrents.php?action=advanced&searchstr=%s&scene=%s&filter_cat[%d]=1',
+        'download': 'https://torrentshack.net/%s',
+    }
+
+    http_time_between_calls = 1  # Seconds
+
+    def _search(self, media, quality, results):
+
+        url = self.urls['search'] % self.buildUrl(media, quality)
+        data = self.getHTMLData(url)
+
+        if data:
+            html = BeautifulSoup(data)
+
+            try:
+                result_table = html.find('table', attrs = {'id': 'torrent_table'})
+                if not result_table:
+                    return
+
+                entries = result_table.find_all('tr', attrs = {'class': 'torrent'})
+
+                for result in entries:
+
+                    link = result.find('span', attrs = {'class': 'torrent_name_link'}).parent
+                    url = result.find('td', attrs = {'class': 'torrent_td'}).find('a')
+
+                    results.append({
+                        'id': link['href'].replace('torrents.php?torrentid=', ''),
+                        'name': six.text_type(link.span.string).translate({ord(six.u('\xad')): None}),
+                        'url': self.urls['download'] % url['href'],
+                        'detail_url': self.urls['download'] % link['href'],
+                        'size': self.parseSize(result.find_all('td')[4].string),
+                        'seeders': tryInt(result.find_all('td')[6].string),
+                        'leechers': tryInt(result.find_all('td')[7].string),
+                    })
+
+            except:
+                log.error('Failed to parsing %s: %s', (self.getName(), traceback.format_exc()))
+
+    def getLoginParams(self):
+        return {
+            'username': self.conf('username'),
+            'password': self.conf('password'),
+            'keeplogged': '1',
+            'login': 'Login',
+        }
+
+    def loginSuccess(self, output):
+        return 'logout.php' in output.lower()
+
+    loginCheckSuccess = loginSuccess
+
+    def getSceneOnly(self):
+        return '1' if self.conf('scene_only') else ''
+
+
+config = [{
+    'name': 'torrentshack',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'TorrentShack',
+            'description': 'See <a href="https://www.torrentshack.net/">TorrentShack</a>',
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False,
+                },
+                {
+                    'name': 'username',
+                    'default': '',
+                },
+                {
+                    'name': 'password',
+                    'default': '',
+                    'type': 'password',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'scene_only',
+                    'type': 'bool',
+                    'default': False,
+                    'description': 'Only allow scene releases.'
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        },
+    ],
+}]

couchpotato/core/media/_base/providers/torrent/torrentz.py

@@ -0,0 +1,128 @@
+import re
+import traceback
+
+from couchpotato.core.helpers.encoding import tryUrlencode
+from couchpotato.core.helpers.rss import RSS
+from couchpotato.core.helpers.variable import tryInt, splitString
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentMagnetProvider
+import six
+
+
+log = CPLog(__name__)
+
+
+class Base(TorrentMagnetProvider, RSS):
+
+    urls = {
+         'detail': 'https://torrentz.eu/%s',
+         'search': 'https://torrentz.eu/feed?q=%s',
+         'verified_search': 'https://torrentz.eu/feed_verified?q=%s'
+    }
+
+    http_time_between_calls = 0
+
+    def _search(self, media, quality, results):
+
+        search_url = self.urls['verified_search'] if self.conf('verified_only') else self.urls['search']
+
+        # Create search parameters
+        search_params = self.buildUrl(media)
+
+        smin = quality.get('size_min')
+        smax = quality.get('size_max')
+        if smin and smax:
+            search_params += ' size %sm - %sm' % (smin, smax)
+
+        min_seeds = tryInt(self.conf('minimal_seeds'))
+        if min_seeds:
+            search_params += ' seed > %s' % (min_seeds - 1)
+
+        rss_data = self.getRSSData(search_url % search_params)
+
+        if rss_data:
+            try:
+
+                for result in rss_data:
+
+                    name = self.getTextElement(result, 'title')
+                    detail_url = self.getTextElement(result, 'link')
+                    description = self.getTextElement(result, 'description')
+
+                    magnet = splitString(detail_url, '/')[-1]
+                    magnet_url = 'magnet:?xt=urn:btih:%s&dn=%s&tr=%s' % (magnet.upper(), tryUrlencode(name), tryUrlencode('udp://tracker.openbittorrent.com/announce'))
+
+                    reg = re.search('Size: (?P<size>\d+) MB Seeds: (?P<seeds>[\d,]+) Peers: (?P<peers>[\d,]+)', six.text_type(description))
+                    size = reg.group('size')
+                    seeds = reg.group('seeds').replace(',', '')
+                    peers = reg.group('peers').replace(',', '')
+
+                    results.append({
+                        'id': magnet,
+                        'name': six.text_type(name),
+                        'url': magnet_url,
+                        'detail_url': detail_url,
+                        'size': tryInt(size),
+                        'seeders': tryInt(seeds),
+                        'leechers': tryInt(peers),
+                    })
+
+            except:
+                log.error('Failed getting results from %s: %s', (self.getName(), traceback.format_exc()))
+
+
+config = [{
+    'name': 'torrentz',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'Torrentz',
+            'description': 'Torrentz is a free, fast and powerful meta-search engine. <a href="https://torrentz.eu/">Torrentz</a>',
+            'wizard': True,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False
+                },
+                {
+                    'name': 'verified_only',
+                    'type': 'bool',
+                    'default': True,
+                    'advanced': True,
+                    'description': 'Only search verified releases',
+                },
+                {
+                    'name': 'minimal_seeds',
+                    'type': 'int',
+                    'default': 1,
+                    'advanced': True,
+                    'description': 'Only return releases with minimal X seeds',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        }
+    ]
+}]

couchpotato/core/media/_base/providers/torrent/yify.py

@@ -0,0 +1,117 @@
+import traceback
+
+from couchpotato.core.helpers.variable import tryInt, getIdentifier
+from couchpotato.core.logger import CPLog
+from couchpotato.core.media._base.providers.torrent.base import TorrentProvider
+
+
+log = CPLog(__name__)
+
+
+class Base(TorrentProvider):
+
+    urls = {
+        'test': '%s/api',
+        'search': '%s/api/list.json?keywords=%s&quality=%s',
+        'detail': '%s/api/movie.json?id=%s'
+    }
+
+    http_time_between_calls = 1  # seconds
+
+    proxy_list = [
+        'http://yify.unlocktorrent.com',
+        'http://yify-torrents.com.come.in',
+        'http://yts.re',
+        'http://yts.im'
+        'http://yify-torrents.im',
+    ]
+
+    def search(self, movie, quality):
+
+        if not quality.get('hd', False):
+            return []
+
+        return super(Base, self).search(movie, quality)
+
+    def _search(self, movie, quality, results):
+
+        search_url = self.urls['search'] % (self.getDomain(), getIdentifier(movie), quality['identifier'])
+
+        data = self.getJsonData(search_url)
+
+        if data and data.get('MovieList'):
+            try:
+                for result in data.get('MovieList'):
+
+                    try:
+                        title = result['TorrentUrl'].split('/')[-1][:-8].replace('_', '.').strip('._')
+                        title = title.replace('.-.', '-')
+                        title = title.replace('..', '.')
+                    except:
+                        continue
+
+                    results.append({
+                        'id': result['MovieID'],
+                        'name': title,
+                        'url': result['TorrentMagnetUrl'],
+                        'detail_url': self.urls['detail'] % (self.getDomain(), result['MovieID']),
+                        'size': self.parseSize(result['Size']),
+                        'seeders': tryInt(result['TorrentSeeds']),
+                        'leechers': tryInt(result['TorrentPeers'])
+                    })
+
+            except:
+                log.error('Failed getting results from %s: %s', (self.getName(), traceback.format_exc()))
+
+    def correctProxy(self, data):
+        data = data.lower()
+        return 'yify' in data and 'yts' in data
+
+
+config = [{
+    'name': 'yify',
+    'groups': [
+        {
+            'tab': 'searcher',
+            'list': 'torrent_providers',
+            'name': 'Yify',
+            'description': 'Free provider, less accurate. Small HD movies, encoded by <a href="https://yify-torrents.com/">Yify</a>.',
+            'wizard': False,
+            'options': [
+                {
+                    'name': 'enabled',
+                    'type': 'enabler',
+                    'default': False
+                },
+                {
+                    'name': 'domain',
+                    'advanced': True,
+                    'label': 'Proxy server',
+                    'description': 'Domain for requests, keep empty to let CouchPotato pick.',
+                },
+                {
+                    'name': 'seed_ratio',
+                    'label': 'Seed ratio',
+                    'type': 'float',
+                    'default': 1,
+                    'description': 'Will not be (re)moved until this seed ratio is met.',
+                },
+                {
+                    'name': 'seed_time',
+                    'label': 'Seed time',
+                    'type': 'int',
+                    'default': 40,
+                    'description': 'Will not be (re)moved until this seed time (in hours) is met.',
+                },
+                {
+                    'name': 'extra_score',
+                    'advanced': True,
+                    'label': 'Extra Score',
+                    'type': 'int',
+                    'default': 0,
+                    'description': 'Starting score for each release found via this provider.',
+                }
+            ],
+        }
+    ]
+}]

couchpotato/core/media/_base/providers/userscript/base.py

@@ -1,9 +1,11 @@
+from urlparse import urlparse
+
 from couchpotato.core.event import addEvent, fireEvent
 from couchpotato.core.helpers.encoding import simplifyString
 from couchpotato.core.helpers.variable import getImdb, md5
 from couchpotato.core.logger import CPLog
 from couchpotato.core.plugins.base import Plugin
-from urlparse import urlparse
+
 
 log = CPLog(__name__)
 

couchpotato/core/media/_base/search/__init__.py

@@ -1,7 +1,5 @@
 from .main import Search
 
 
-def start():
+def autoload():
     return Search()
-
-config = []

couchpotato/core/media/_base/search/static/search.css

@@ -6,12 +6,11 @@
 	top: 0;
 	text-align: right;
 	height: 100%;
-	border-bottom: 4px solid transparent;
 	transition: all .4s cubic-bezier(0.9,0,0.1,1);
 	position: absolute;
 	z-index: 20;
-	border: 1px solid transparent;
-	border-width: 0 0 4px;
+	border: 0 solid transparent;
+	border-bottom-width: 4px;
 }
 	.search_form:hover {
 		border-color: #047792;
@@ -22,19 +21,19 @@
 			right: 44px;
 		}
 	}
-	
+
 	.search_form.focused,
 	.search_form.shown {
 		border-color: #04bce6;
 	}
-	
+
 	.search_form .input {
 		height: 100%;
 		overflow: hidden;
 		width: 45px;
 		transition: all .4s cubic-bezier(0.9,0,0.1,1);
 	}
-	
+
 		.search_form.focused .input,
 		.search_form.shown .input  {
 			width: 380px;
@@ -49,7 +48,6 @@
 				color: #FFF;
 				font-size: 25px;
 				height: 100%;
-				padding: 10px;
 				width: 100%;
 				opacity: 0;
 				padding: 0 40px 0 10px;
@@ -59,23 +57,23 @@
 				.search_form.shown .input input {
 					opacity: 1;
 				}
-				
+
 				.search_form input::-ms-clear {
 				    width : 0;
 					height: 0;
 				}
-					
+
 			@media all and (max-width: 480px) {
 				.search_form .input input {
 					font-size: 15px;
 				}
-				
+
 				.search_form.focused .input,
 				.search_form.shown .input {
 					width: 277px;
 				}
 			}
-	
+
 	.search_form .input a  {
 		position: absolute;
 		top: 0;
@@ -89,7 +87,7 @@
 		font-size: 15px;
 		color: #FFF;
 	}
-		
+
 		.search_form .input a:after {
 			content: "\e03e";
 		}
@@ -97,7 +95,7 @@
 		.search_form.shown.filled .input a:after {
 			content: "\e04e";
 		}
-		
+
 		@media all and (max-width: 480px) {
 			.search_form .input a  {
 				line-height: 44px;
@@ -167,13 +165,13 @@
 				.media_result .options select[name=title] { width: 170px; }
 				.media_result .options select[name=profile] { width: 90px; }
 				.media_result .options select[name=category] { width: 80px; }
-				
+
 				@media all and (max-width: 480px) {
-				
+
 					.media_result .options select[name=title] { width: 90px; }
 					.media_result .options select[name=profile] { width: 50px; }
 					.media_result .options select[name=category] { width: 50px; }
-					
+
 				}
 
 			.media_result .options .button {
@@ -227,14 +225,14 @@
 				right: 7px;
 				vertical-align: middle;
 			}
-			
+
 				.media_result .info h2 {
 					margin: 0;
 					font-weight: normal;
 					font-size: 20px;
 					padding: 0;
 				}
-				
+
 					.search_form .info h2 {
 						position: absolute;
 						width: 100%;
@@ -247,12 +245,12 @@
 					overflow: hidden;
 					white-space: nowrap;
 				}
-				
+
 					.search_form .info h2 .title {
 						position: absolute;
 						width: 88%;
 					}
-	
+
 					.media_result .info h2 .year {
 						padding: 0 5px;
 						text-align: center;
@@ -260,14 +258,14 @@
 						width: 12%;
 						right: 0;
 					}
-					
+
 					@media all and (max-width: 480px) {
-					
+
 						.search_form .info h2 .year {
 							font-size: 12px;
 							margin-top: 7px;
 						}
-						
+
 					}
 
 .search_form .mask,
@@ -277,4 +275,4 @@
 	width: 100%;
 	left: 0;
 	top: 0;
-}
+}

couchpotato/core/media/_base/search/static/search.js

@@ -16,7 +16,7 @@ Block.Search = new Class({
 						'keyup': self.keyup.bind(self),
 						'focus': function(){
 							if(focus_timer) clearTimeout(focus_timer);
-							self.el.addClass('focused')
+							self.el.addClass('focused');
 							if(this.get('value'))
 								self.hideResults(false)
 						},
@@ -57,17 +57,17 @@ Block.Search = new Class({
 		(e).preventDefault();
 
 		if(self.last_q === ''){
-			self.input.blur()
+			self.input.blur();
 			self.last_q = null;
 		}
 		else {
 
 			self.last_q = '';
 			self.input.set('value', '');
-			self.input.focus()
+			self.input.focus();
 
-			self.media = {}
-			self.results.empty()
+			self.media = {};
+			self.results.empty();
 			self.el.removeClass('filled')
 
 		}
@@ -92,16 +92,16 @@ Block.Search = new Class({
 		self.hidden = bool;
 	},
 
-	keyup: function(e){
+	keyup: function(){
 		var self = this;
 
-		self.el[self.q() ? 'addClass' : 'removeClass']('filled')
+		self.el[self.q() ? 'addClass' : 'removeClass']('filled');
 
 		if(self.q() != self.last_q){
 			if(self.api_request && self.api_request.isRunning())
 				self.api_request.cancel();
 
-			if(self.autocomplete_timer) clearTimeout(self.autocomplete_timer)
+			if(self.autocomplete_timer) clearTimeout(self.autocomplete_timer);
 			self.autocomplete_timer = self.autocomplete.delay(300, self)
 		}
 
@@ -111,7 +111,7 @@ Block.Search = new Class({
 		var self = this;
 
 		if(!self.q()){
-			self.hideResults(true)
+			self.hideResults(true);
 			return
 		}
 
@@ -139,7 +139,7 @@ Block.Search = new Class({
 			})
 		}
 		else
-			self.fill(q, cache)
+			self.fill(q, cache);
 
 		self.last_q = q;
 
@@ -148,31 +148,31 @@ Block.Search = new Class({
 	fill: function(q, json){
 		var self = this;
 
-		self.cache[q] = json
+		self.cache[q] = json;
 
-		self.media = {}
-		self.results.empty()
-		
-		Object.each(json, function(media, type){
+		self.media = {};
+		self.results.empty();
+
+		Object.each(json, function(media){
 			if(typeOf(media) == 'array'){
 				Object.each(media, function(m){
-		
+
 					var m = new Block.Search[m.type.capitalize() + 'Item'](m);
-					$(m).inject(self.results)
-					self.media[m.imdb || 'r-'+Math.floor(Math.random()*10000)] = m
-		
+					$(m).inject(self.results);
+					self.media[m.imdb || 'r-'+Math.floor(Math.random()*10000)] = m;
+
 					if(q == m.imdb)
 						m.showOptions()
-		
+
 				});
 			}
-		})
+		});
 
 		// Calculate result heights
 		var w = window.getSize(),
 			rc = self.result_container.getCoordinates();
 
-		self.results.setStyle('max-height', (w.y - rc.top - 50) + 'px')
+		self.results.setStyle('max-height', (w.y - rc.top - 50) + 'px');
 		self.mask.fade('out')
 
 	},
@@ -185,4 +185,4 @@ Block.Search = new Class({
 		return this.input.get('value').trim();
 	}
 
-});
+});

couchpotato/core/media/_base/searcher/__init__.py

@@ -1,7 +1,7 @@
 from .main import Searcher
 
 
-def start():
+def autoload():
     return Searcher()
 
 config = [{

couchpotato/core/media/_base/searcher/main.py

@@ -1,20 +1,24 @@
+import datetime
+import re
+
 from couchpotato.api import addApiView
 from couchpotato.core.event import addEvent, fireEvent
 from couchpotato.core.helpers.encoding import simplifyString
 from couchpotato.core.helpers.variable import splitString, removeEmpty, removeDuplicate
 from couchpotato.core.logger import CPLog
 from couchpotato.core.media._base.searcher.base import SearcherBase
-import datetime
-import re
+
 
 log = CPLog(__name__)
 
 
 class Searcher(SearcherBase):
 
+    # noinspection PyMissingConstructor
     def __init__(self):
         addEvent('searcher.protocols', self.getSearchProtocols)
         addEvent('searcher.contains_other_quality', self.containsOtherQuality)
+        addEvent('searcher.correct_3d', self.correct3D)
         addEvent('searcher.correct_year', self.correctYear)
         addEvent('searcher.correct_name', self.correctName)
         addEvent('searcher.correct_words', self.correctWords)
@@ -48,7 +52,7 @@ class Searcher(SearcherBase):
         results = []
 
         for search_protocol in protocols:
-            protocol_results = fireEvent('provider.search.%s.%s' % (search_protocol, media['type']), media, quality, merge = True)
+            protocol_results = fireEvent('provider.search.%s.%s' % (search_protocol, media.get('type')), media, quality, merge = True)
             if protocol_results:
                 results += protocol_results
 
@@ -121,6 +125,17 @@ class Searcher(SearcherBase):
 
         return not (found.get(preferred_quality['identifier']) and len(found) == 1)
 
+    def correct3D(self, nzb, preferred_quality = None):
+        if not preferred_quality: preferred_quality = {}
+        if not preferred_quality.get('custom'): return
+
+        threed = preferred_quality['custom'].get('3d')
+
+        # Try guessing via quality tags
+        guess = fireEvent('quality.guess', [nzb.get('name')], single = True)
+
+        return threed == guess.get('is_3d')
+
     def correctYear(self, haystack, year, year_range):
 
         if not isinstance(haystack, (list, tuple, set)):
@@ -181,7 +196,7 @@ class Searcher(SearcherBase):
             req = splitString(req_set, '&')
             req_match += len(list(set(rel_words) & set(req))) == len(req)
 
-        if len(required_words) > 0  and req_match == 0:
+        if len(required_words) > 0 and req_match == 0:
             log.info2('Wrong: Required word missing: %s', rel_name)
             return False