paperboy

https://img.shields.io/badge/Status-BETA%201-yellow.svg?&longCache=true&style=for-the-badgeStatus

A web frontend for scheduling Jupyter Notebooks as reports

Overview

https://dev.azure.com/tpaine154/jupyter/_apis/build/status/timkpaine.paperboy?branchName=masterBuild Status https://img.shields.io/azure-devops/coverage/tpaine154/jupyter/9Azure DevOps coverage https://img.shields.io/readthedocs/paperboy.svgDocs https://img.shields.io/badge/Site--grey.svg?colorB=FFFFFFSite

Paperboy is a production-grade application for scheduling reports. It has a flexible architecture and extensible APIs, and can integrate into a wide variety of deployments. It is composed of various industrial-strength technologies from the open source world.

  • Jupyter Notebooks for the reports themselves
    • Jupyter notebooks are an ideal report template, and with NBConvert support a wide variety of output types, including PDFs, HTML, Emails, etc
  • Papermill to parameterize notebooks
  • SQLAlchemy for Storage (default)
  • Apache Airflow for Scheduling (default)
  • PhosphorJS for the frontend
  • Support for Python Virtualenvs via requirements.txt or custom Docker images via Dockerfiles on a per-notebook level
  • Traitlets parameterization of storage and scheduler classes for easy integration with custom storage backends and custom schedulers
  • Single click notebook deployment with Voila and Dokku

https://raw.githubusercontent.com/timkpaine/paperboy/master/docs/img/ss.png

Process Flow

  • Upload notebook
  • Configure job
    • start time
    • interval
    • papermill parameters to autoconfigure reports
    • if autoconfiguring reports from papermill:
      • run or publish
      • output
        • notebook
        • pdf
        • html
        • email
        • script
      • strip or keep code
  • To edit or create additional reports on a job, configure reports
    • run or publish
    • output
      • notebook
      • pdf
      • html
      • email
      • script
    • strip or keep code

Installation from source

Paperboy requires Python and Node.js, which can be installed from conda-forge if conda is available.

Clone the repository and run following commands to install and launch the application:

  • npm install
  • npm run build
  • pip install -e .
  • python -m paperboy

Visit http://0.0.0.0:8080 in a browser to view the application.

The default authentication backend requires the registration of a username that can be used on subsequent launches.

API Documentation

paperboy.server.main()[source]

Use gunicorn on linux or waitress on windows to deplot a paperboy instance

paperboy.server.api.FalconAPI(config)[source]

Create falcon.API application from Paperboy traitlets application

Parameters:config (paperboy.config.application.Paperboy) – traitlets configuration for application
Returns:the routed falcon api to launch with gunicorn/waitress
Return type:falcon.API
class paperboy.storage.sqla.user.UserSQLStorage(*args, **kwargs)[source]

Bases: paperboy.storage.sqla.base.BaseSQLStorageMixin, paperboy.storage.base.UserStorage

delete(user, params, session, *args, **kwargs)[source]

Method to delete instance

detail(user, params, session, *args, **kwargs)[source]

token -> user

form()[source]

Pass through to shared method in BaseSQLStorageMixin

list(user, params, session, *args, **kwargs)[source]

Not used

login(user, params, session, *args, **kwargs)[source]

username/password -> user/token

search(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

status(*args, **kwargs)[source]

Not used

store(user, params, session, *args, **kwargs)[source]

Method to save/update instance

class paperboy.storage.sqla.job.JobSQLStorage(config, *args, **kwargs)[source]

Bases: paperboy.storage.sqla.base.BaseSQLStorageMixin, paperboy.storage.base.JobStorage

delete(user, params, session, scheduler, *args, **kwargs)[source]

Method to delete instance

detail(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

form()[source]

Pass through to shared method in BaseSQLStorageMixin

list(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

search(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

status(user, params, session, *args, **kwargs)[source]

Method to populate general status information in Status tab of gui

store(user, params, session, scheduler, *args, **kwargs)[source]

Method to save/update instance

class paperboy.storage.sqla.models.user.UserSQL(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

id
jobs
name
notebooks
password
reports
class paperboy.storage.sqla.models.job.JobSQL(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

created
static from_config(jb)[source]
id
interval
level
modified
name
notebook
notebookId
reports
start_time
to_config(config)[source]
user
userId
class paperboy.storage.sqla.models.notebook.NotebookSQL(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

created
dockerfile
static from_config(nb)[source]
id
jobs
level
modified
name
notebook
privacy
reports
requirements
to_config(config)[source]
user
userId
class paperboy.storage.sqla.models.report.ReportSQL(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

created
static from_config(rp)[source]
id
job
jobId
modified
name
notebook
notebookId
output
parameters
strip_code
to_config(config)[source]
type
user
userId
class paperboy.storage.sqla.models.output.OutputSQL(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

created
data = LargeBinary()
id
name
report
reportId
class paperboy.storage.sqla.notebook.NotebookSQLStorage(config, *args, **kwargs)[source]

Bases: paperboy.storage.sqla.base.BaseSQLStorageMixin, paperboy.storage.base.NotebookStorage

delete(user, params, session, *args, **kwargs)[source]

Method to delete instance

detail(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

form()[source]

Pass through to shared method in BaseSQLStorageMixin

list(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

search(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

status(user, params, session, *args, **kwargs)[source]

Method to populate general status information in Status tab of gui

store(user, params, session, *args, **kwargs)[source]

Method to save/update instance

class paperboy.storage.sqla.base.BaseSQLStorageMixin[source]

Bases: object

Base class to share common SQL operations between User/Notebook/Job/Report backends

paperboy.storage.sqla.base.justid(id)[source]

Helper method to turn name-id string into just id

paperboy.storage.sqla.base.lookfor(s)[source]

Helpful matcher for sqlalchemy query

class paperboy.storage.sqla.report.ReportSQLStorage(config, *args, **kwargs)[source]

Bases: paperboy.storage.sqla.base.BaseSQLStorageMixin, paperboy.storage.base.ReportStorage

delete(user, params, session, *args, **kwargs)[source]

Method to delete instance

detail(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

form()[source]

Pass through to shared method in BaseSQLStorageMixin

generate(user, params, session, *args, **kwargs)[source]

Generate reports from Job instance and JSONL papermill parameters

list(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

search(user, params, session, *args, **kwargs)[source]

Pass through to shared method in BaseSQLStorageMixin

status(user, params, session, *args, **kwargs)[source]

Method to populate general status information in Status tab of gui

store(user, params, session, *args, **kwargs)[source]

Method to save/update instance

paperboy.storage.sqla.fixtures.main(sql_url)[source]

Create dummy notebook data for sqlalchemy

paperboy.storage.utils.strip_outputs(nb)[source]

Helper wrapper around nbstripout to remove notebook outputs

class paperboy.storage.manager.StorageEngine(user_storage, notebook_storage, job_storage, report_storage)[source]

Bases: object

Unified interface into storage classes of a backend

add_thing(thing)[source]
get_things(marker, limit)[source]
jobs
notebooks
reports
users
exception paperboy.storage.manager.StorageError[source]

Bases: Exception

static handle(ex, req, resp, params)[source]
class paperboy.storage.base.BaseStorage(config, *args, **kwargs)[source]

Bases: object

Abstract base class representing the interface to a storage configuration

delete(*args, **kwargs)[source]

Method to delete instance

detail(*args, **kwargs)[source]

Method to view detailed fields about type

form(*args, **kwargs)[source]

Method to generate form on gui to create new instance.

Returns:paperboy.config.forms.Response as dict
list(*args, **kwargs)[source]

Method to list all instances of type

search(*args, **kwargs)[source]

Method to search for instance

status(*args, **kwargs)[source]

Method to populate general status information in Status tab of gui

store(*args, **kwargs)[source]

Method to save/update instance

class paperboy.storage.base.JobStorage(config, *args, **kwargs)[source]

Bases: paperboy.storage.base.BaseStorage

Base class for Job storage. Job backends should inherit from this class

class paperboy.storage.base.NotebookStorage(config, *args, **kwargs)[source]

Bases: paperboy.storage.base.BaseStorage

Base class for Notebook storage. Notebook backends should inherit from this class

class paperboy.storage.base.ReportStorage(config, *args, **kwargs)[source]

Bases: paperboy.storage.base.BaseStorage

Base class for Report storage. Report backends should inherit from this class

generate(*args, **kwargs)[source]

Generate reports from Job instance and JSONL papermill parameters

class paperboy.storage.base.UserStorage(config, *args, **kwargs)[source]

Bases: paperboy.storage.base.BaseStorage

login(*args, **kwargs)[source]

Handler for user login

logout(*args, **kwargs)[source]

Handler for user logout

paperboy.utils.class_to_name(cls)[source]
paperboy.utils.name_to_class(name)[source]

Copied from https://github.com/yohanboniface/falcon-multipart due to conda-forge issues.

MIT License

Copyright (c) 2017 Yohan Boniface

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

class paperboy.middleware.multipart.MultipartMiddleware(parser=None)[source]

Bases: object

parse(stream, environ)[source]
parse_field(field)[source]
process_request(req, resp, **kwargs)[source]
class paperboy.middleware.multipart.Parser(fp=None, headers=None, outerboundary=b'', environ=environ({'HOSTNAME': 'build-10518502-project-246625-paperboy', 'PYPY_VERSION_35': 'pypy3.5-7.0.0', 'APPDIR': '/app', 'HOME': '/home/docs', 'OLDPWD': '/', 'CONDA_VERSION': '4.7.12.1', 'READTHEDOCS': 'True', 'READTHEDOCS_PROJECT': 'paperboy', 'PATH': '/home/docs/checkouts/readthedocs.org/user_builds/paperboy/envs/latest/bin:/home/docs/.pyenv/shims:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/docs/.conda/bin:/home/docs/.pyenv/bin', 'LANG': 'C.UTF-8', 'READTHEDOCS_LANGUAGE': 'en', 'DEBIAN_FRONTEND': 'noninteractive', 'PYTHON_VERSION_35': '3.5.7', 'PYTHON_VERSION_27': '2.7.16', 'PYTHON_VERSION_36': '3.6.8', 'READTHEDOCS_VERSION': 'latest', 'PWD': '/home/docs/checkouts/readthedocs.org/user_builds/paperboy/checkouts/latest/docs', 'PYTHON_VERSION_37': '3.7.3', 'PYENV_ROOT': '/home/docs/.pyenv', 'PYTHON_VERSION_38': '3.8.0', 'DOCUTILSCONFIG': '/home/docs/checkouts/readthedocs.org/user_builds/paperboy/checkouts/latest/docs/docutils.conf'}), keep_blank_values=0, strict_parsing=0, limit=None, encoding='utf-8', errors='replace', max_num_fields=None)[source]

Bases: cgi.FieldStorage

class paperboy.middleware.none.NoAuthRequiredMiddleware(config, db, *args, **kwargs)[source]

Bases: object

Dummy no-auth-required middleware

process_resource(req, resp, resource, params)[source]
class paperboy.middleware.none.NoUserMiddleware(config, db, *args, **kwargs)[source]

Bases: object

Dummy user authentication middleware

process_request(req, resp)[source]

inject anonymous user into every context

Copied from https://github.com/lwcolton/falcon-cors due to conda-forge issues.

Apache License

Version 2.0, January 2004

http://www.apache.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

  1. Definitions.

    “License” shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.

    “Licensor” shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.

    “Legal Entity” shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, “control” means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.

    “You” (or “Your”) shall mean an individual or Legal Entity exercising permissions granted by this License.

    “Source” form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.

    “Object” form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.

    “Work” shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).

    “Derivative Works” shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.

    “Contribution” shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, “submitted” means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as “Not a Contribution.”

    “Contributor” shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.

  2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.

  3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.

  4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:

    1. You must give any other recipients of the Work or Derivative Works a copy of this License; and
    2. You must cause any modified files to carry prominent notices stating that You changed the files; and
    3. You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
    4. If the Work includes a “NOTICE” text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.

    You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.

  5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.

  6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.

  7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.

  8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.

  9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.

END OF TERMS AND CONDITIONS

APPENDIX: How to apply the Apache License to your work.

To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets “{}” replaced with your own identifying information. (Don’t include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same “printed page” as the copyright notice for easier identification within third-party archives.

Copyright {yyyy} {name of copyright owner}

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

class paperboy.middleware.cors.CORS(**cors_config)[source]

Bases: object

Initialize a CORS object, passing in configuration options. All of the configuration settings are optional, however if none of them are specified the default configuration will simply deny all CORS requests. You can pass this to API for a global configuration. After enabling globally, you can override the settings for a particular resource by setting the ‘cors’ attribute on it to an instance of this class. :param logger: Specifies the logger to use. A basic logger and StreamHandler

will be configure for you if none is provided.
Parameters:
  • allow_all_origins (bool, optional) – Specifies whether CORS should allow requests from all origins. Default is False.
  • allow_origins_list (list, optional) – A list of origins that are allowed to make CORS requests. Default is empty.
  • allow_origins_regex (str, optional) – A string containing a Python regular expression that matches origins which are allowed to make CORS requests. Default is None.
  • allow_all_headers (bool, optional) – If True, when the server is responding to a preflight request it will approve any headers requested by the client via the Access-Control-Request-Headers header, setting each requested header in the value of the Access-Control-Allow-Headers header in the response. Default is False.
  • allow_headers_list (list, optional) – A list of headers which are allowed values for the Access-Control-Allow-Headers header in response to a preflight request. When the server is responding to a preflight request, it will check each header requested by the client in the Access-Control-Request-Headers header to see if it exists in this list. If it does, it will be included in the Access-Control-Allow-Headers header in the response to the preflight request. Default is empty.
  • allow_headers_regex (str, optional) – A string containing a Python regular expression that matches headers that should be allowed in response to a preflight request. If this is set, when a preflight request is received by the server, it will try to match each header requested by the client via the Access-Control-Request-Headers header of the request. If the requested header is matched by this regex, it will be included in the value of the Access-Control-Allow-Headers header of the response.
  • expose_headers_list (list, optional) – A list of headers that should be sent as values to the Access-Control-Expose-Headers header in response to simple or actual requests.
  • allow_all_methods (bool, optional) – Specifies whether all methods are allowed via CORS requests. Default is False.
  • allow_methods_list (list, optional) – A list of methods which are allowed via CORS requests. These should be values from falcon.HTTP_METHODS, which are strings like ‘GET’ and ‘PATCH’. Default is empty.
  • allow_credentials_all_origins (bool, optional) – Where or not the Access-Control-Allow-Credentials should be set to True and set on all responses. Default is False.
  • allow_credentials_origins_list (list, optional) – A list of origins for which the Access-Control-Allow-Credentials header should be set to True and included with all responses. Default is empty.
  • allow_credentials_origins_regex (string, optional) – A string containing a Python regular expression matching origins for which the Access-Control-Allow-Credentials header should be set to True and included in all responses. Default is None.
  • max_age (int, optional) – If set to an integer, this value will be used as the value of the Access-Control-Max-Age header in response to preflight requests. This is in seconds the maximum amount of time a client may cache responses to preflight requests. Default is None (no header sent).

Note

The arguments above are inclusie, meaning a header, origin, or method will only be disallowed if it doesn’t match ANY specification. First the allow_all directive is checked, then the list directive, then the regex directive if applicable, then list by method if applicable, and lastly regex by method if applicable. For instance, this means if you specify ‘Auth-Key’ in allow_headers_list, it will be allowed for all methods regardless of the values in header_list_By_method.

Note

Headers are converted to lower-case for you. Methods are converted to upper-case for you. Take note of this if you are writing regular expressions.

Note

The allow_headers_* settings relate to the Access-Control-Allow-Headers header which is only sent in response to pre-flight requests. This is different from the Access-Control-Expose-Headers header which is set via the expose_headers_list setting and is sent only in response to basic or actual requests.

Warning

Exercise caution when using the regex enabled settings. It is very easy to misunderstand Python regex syntax and accidentally introduce an unintentionally allowed origin or other vulnerability into your application.

middleware

A property which returns a CORSMiddleware instance

process(req, resp, resource)[source]
class paperboy.middleware.cors.CORSMiddleware(cors, default_enabled=True)[source]

Bases: object

This is the middleware that applies a CORS object to requests. :param cors: An instance of CORS. :type cors: CORS, required :param default_enabled: Whether CORS processing should

take place for every resource. Default True.
process_resource(req, resp, resource, *args)[source]
paperboy.middleware.cors.get_default_logger(level=None)[source]

Copied from https://gitlab.com/skosh/falcon-helpers due to conda-forge issues Copyright (c) 2017 by Nicholas Zaccardi

Some rights reserved.

Redistribution and use in source and binary forms of the software as well as documentation, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  • The names of the contributors may not be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE AND DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE AND DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

class paperboy.middleware.load_user.LoadUserMiddleware(user_cls, get_id=None, session=None)[source]

Bases: object

Load a user from the database during the request cycle using sqlalchemy

By default this will grab the id data from the auth token on the request context assuming you are using .middleware.auth_required. To change that pass a function which takes the request and returns the user id to get_id.

Parameters:
  • session – a sqlalchemy session
  • user_cls – a class which will return the user object. This object must have a get_by_id which will return a SQLAlchemy Query.
  • get_id – a function which will get the user identifier off of the request.
fetch_user(user_id)[source]
process_request(req, resp)[source]
class paperboy.middleware.sqla.SQLAlchemySessionMiddleware(sessionmaker=None)[source]

Bases: object

variant of https://gitlab.com/skosh/falcon-helpers/blob/master/falcon_helpers/middlewares/sqla.py

process_resource(req, resp, resource, params)[source]

initialize SQL Alchemy session and put into resource’s session variable

process_response(req, resp, resource, req_succeeded)[source]

If session is successful, commit, otherwise revert

class paperboy.middleware.sqla.SQLAuthRequiredMiddleware(config, db)[source]

Bases: object

Middleware to authenticate user with sqlalchemy

process_request(req, resp)[source]

if auth token present, put into context

process_resource(req, resp, resource, params)[source]
class paperboy.middleware.sqla.SQLUserMiddleware(config, db, *args, **kwargs)[source]

Bases: object

Middleware to fetch current user and put in resource’s context

process_request(req, resp)[source]

Get user from auth token

Copied from https://gitlab.com/skosh/falcon-helpers due to conda-forge issues Copyright (c) 2017 by Nicholas Zaccardi

Some rights reserved.

Redistribution and use in source and binary forms of the software as well as documentation, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  • The names of the contributors may not be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE AND DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE AND DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

class paperboy.middleware.auth_required.AuthRequiredMiddleware(resource_param='auth_required', context_key='auth_token_contents', when_fails=<function _default_failed>)[source]

Bases: object

Requires a cookie be set with a valid JWT or fails

Example

import falcon from falcon_helpers.middlewares.auth_required import AuthRequiredMiddleware

class Resource:

auth_required = True

def on_get(self, req, resp):
# …
def when_fails_auth(req, resp, token_value):
raise TerribleException(token_value)
api = falcon.API(
middleware=[
AuthRequiredMiddleware(when_fails=when_fails_auth)

]

)

api.add_route(‘/’, Resource())

resource_param

The paramater to pull the boolean from

context_key

the key the token will be found on the request

when_fails

(function) A function to execute when the authentication fails

process_resource(req, resp, resource, params)[source]
class paperboy.config.user.UserConfig(config, *args, **kwargs)[source]

Bases: paperboy.config.base.Base

Paperboy configuration object representing a User

edit()[source]

Generate Edit template for client from a User object

form()[source]

Generate Form template for client from a User object

static from_json(jsn, config)[source]

Create User from a JSON

id

A trait for unicode strings.

name

A trait for unicode strings.

store()[source]

Generate response modal for client when saving a User object

to_json()[source]

Convert User to a JSON

class paperboy.config.job.JobConfig(config, *args, **kwargs)[source]

Bases: paperboy.config.base.Base

Paperboy configuration object representing a Job

edit()[source]

Generate Edit template for client from a Job object

entry()[source]

Generate ListTable entry for client from a Job object

form()[source]

Generate Form template for client from a Job object

static from_json(jsn, config)[source]

Create Job from a JSON

id

A trait for unicode strings.

meta

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

name

A trait for unicode strings.

store()[source]

Generate response modal for client when saving a Job object

to_json(include_notebook=False)[source]

Convert Job to a JSON

class paperboy.config.job.JobMetadataConfig(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Paperboy configuration object representing a Job (metadata component)

created

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

static from_json(jsn)[source]

Create JobMetadata from a JSON

interval

A trait for unicode strings.

level

A trait for unicode strings.

modified

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

notebook

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

reports

An int trait.

start_time

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

to_json(include_notebook=False)[source]

Convert JobMetadata to a JSON

userid

A trait for unicode strings.

username

A trait for unicode strings.

class paperboy.config.forms.DOMEntry(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

DOM node template to be rendered on the client

static from_json(jsn)[source]
hidden

A boolean (True, False) trait.

label

A trait for unicode strings.

name

A trait for unicode strings.

options

An instance of a Python list.

placeholder

A trait for unicode strings.

readonly

A boolean (True, False) trait.

required

A boolean (True, False) trait.

to_json()[source]
type

A trait for unicode strings.

value

A trait for unicode strings.

class paperboy.config.forms.FormEntry(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Form template entry to be rendered on the client

hidden

A boolean (True, False) trait.

label

A trait for unicode strings.

name

A trait for unicode strings.

options

An instance of a Python list.

placeholder

A trait for unicode strings.

readonly

A boolean (True, False) trait.

required

A boolean (True, False) trait.

to_json()[source]

Convert form entry to JSON

type

A trait for unicode strings.

url

A trait for unicode strings.

value

A trait for unicode strings.

class paperboy.config.forms.ListResult(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

List result metadata for pagination

count

An int trait.

page

An int trait.

pages

An int trait.

results

An instance of a Python list.

to_json()[source]
total

An int trait.

class paperboy.config.forms.Response(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Response modal template for client

entries

An instance of a Python list.

to_json()[source]
class paperboy.config.storage.SQLAStorageConfig(*args, **kwargs)[source]

Bases: paperboy.config.storage.StorageConfig

Config for SQL Alchemy storage

engine = None
job_storage

alias of paperboy.storage.sqla.job.JobSQLStorage

notebook_storage

alias of paperboy.storage.sqla.notebook.NotebookSQLStorage

report_storage

alias of paperboy.storage.sqla.report.ReportSQLStorage

sessionmaker = None
sql_url

A trait for unicode strings.

sql_user

A boolean (True, False) trait.

type = 'SQLA'
user_storage

alias of paperboy.storage.sqla.user.UserSQLStorage

class paperboy.config.storage.StorageConfig(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Base config for storage backend

job_storage = None
notebook_storage = None
report_storage = None
type

A trait for unicode strings.

user_storage = None
class paperboy.config.scheduler.AirflowSchedulerConfig(*args, **kwargs)[source]

Bases: paperboy.config.scheduler.SchedulerConfig

Configuration for airflow

clazz

alias of paperboy.scheduler.airflow.airflow.AirflowScheduler

config

A trait for unicode strings.

dagbag

A trait for unicode strings.

type = 'airflow'
class paperboy.config.scheduler.DummySchedulerConfig(*args, **kwargs)[source]

Bases: paperboy.config.scheduler.SchedulerConfig

Configuration for airflow

clazz

alias of paperboy.scheduler.base.DummyScheduler

type = 'dummy'
class paperboy.config.scheduler.LocalSchedulerConfig(*args, **kwargs)[source]

Bases: paperboy.config.scheduler.SchedulerConfig

Configuration for local scheduler

clazz

alias of paperboy.scheduler.local.local.LocalScheduler

type = 'local'
working_directory

A trait for unicode strings.

class paperboy.config.scheduler.LuigiSchedulerConfig(*args, **kwargs)[source]

Bases: paperboy.config.scheduler.SchedulerConfig

Configuration for luigi

clazz

alias of paperboy.scheduler.luigi.luigi.LuigiScheduler

crontab

A trait for unicode strings.

db_connection

A trait for unicode strings.

local

A boolean (True, False) trait.

state_path

A trait for unicode strings.

task_folder

A trait for unicode strings.

type = 'luigi'
class paperboy.config.scheduler.SchedulerConfig(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Base config for scheduler

type

A trait for unicode strings.

class paperboy.config.notebook.NotebookConfig(config, *args, **kwargs)[source]

Bases: paperboy.config.base.Base

Paperboy configuration object representing a Notebook

edit()[source]

Generate Edit template for client from a Notebook object

entry()[source]

Generate ListTable entry for client from a Notebook object

form()[source]

Generate Form template for client from a Notebook object

static from_json(jsn, config)[source]

Create Notebook from a JSON

id

A trait for unicode strings.

meta

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

name

A trait for unicode strings.

store()[source]

Generate response modal for client when saving a Notebook object

to_json(include_notebook=False)[source]

Convert Notebook to a JSON

class paperboy.config.notebook.NotebookMetadataConfig(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Paperboy configuration object representing a Notebook (metadata component)

created

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

dockerfile

A trait for unicode strings.

static from_json(jsn)[source]

Create NotebookMetadata from a JSON

jobs

An int trait.

level

A trait for unicode strings.

modified

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

notebook

A trait for unicode strings.

privacy

A trait for unicode strings.

reports

An int trait.

requirements

A trait for unicode strings.

to_json(include_notebook=False)[source]

Convert NotebookMetadata to a JSON

userid

A trait for unicode strings.

username

A trait for unicode strings.

class paperboy.config.base.Base(config, *args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Base HasTraits abstract base class for all paperboy configureables (User, Notebook, Job, and Report)

edit()[source]

generate a JSON edit template for the subclass of Base

Parameters:self – subclass of Base
Returns:python dictionary representing the edit template as a JSON
Return type:dict
entry()[source]
form()[source]

generate a JSON form template for the subclass of Base

Parameters:self – subclass of Base
Returns:python dictionary representing the form template as a JSON
Return type:dict
static from_json(jsn, config)[source]

create a paperboy config object from a json

Parameters:
  • jsn – python dictionary from json representing the configuration object
  • config – paperboy configuration to populate from json
Returns:

subclass of Base populated from jsn

store()[source]
to_json(include_notebook=False)[source]

convert a paperboy config to a json

Parameters:
  • self – subclass of Base
  • include_notebook – if config would include a notebook (potentially several MB in size), should we include it or strip it?
Returns:

python dictionary representing the response json

Return type:

dict

class paperboy.config.base.BaseEnum[source]

Bases: enum.Enum

An enumeration.

members = <bound method BaseEnum.members of <enum 'BaseEnum'>>[source]
values = <bound method BaseEnum.values of <enum 'BaseEnum'>>[source]
class paperboy.config.base.Interval[source]

Bases: paperboy.config.base.BaseEnum

An enumeration.

DAILY = 'daily'
FIVE_MINUTES = '5 minutes'
HOURLY = 'hourly'
MINUTELY = 'minutely'
MONTHLY = 'monthly'
SIX_HOURS = '6 hours'
TEN_MINUTES = '10 minutes'
THIRTY_MINUTES = '30 minutes'
THREE_HOURS = '3 hours'
TWELVE_HOURS = '12 hours'
TWO_HOURS = '2 hours'
WEEKLY = 'weekly'
class paperboy.config.base.OutputType[source]

Bases: paperboy.config.base.BaseEnum

An enumeration.

EMAIL = 'email'
HTML = 'html'
NOTEBOOK = 'notebook'
PDF = 'pdf'
SCRIPT = 'script'
class paperboy.config.base.PrivacyLevel[source]

Bases: paperboy.config.base.BaseEnum

An enumeration.

PRIVATE = 'private'
PUBLIC = 'public'
class paperboy.config.base.ReportType[source]

Bases: paperboy.config.base.BaseEnum

An enumeration.

CONVERT = 'convert'
class paperboy.config.base.ServiceLevel[source]

Bases: paperboy.config.base.BaseEnum

An enumeration.

DEVELOPMENT = 'development'
PERSONAL = 'personal'
PRODUCTION = 'production'
RESEARCH = 'research'
class paperboy.config.report.ReportConfig(config, *args, **kwargs)[source]

Bases: paperboy.config.base.Base

Paperboy configuration object representing a Report

edit()[source]

Generate Edit template for client from a Report object

entry()[source]

Generate ListTable entry for client from a Report object

form()[source]

Generate Form template for client from a Report object

static from_json(jsn, config)[source]

Create Report from a JSON

id

A trait for unicode strings.

meta

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

name

A trait for unicode strings.

store()[source]

Generate response modal for client when saving a Report object

to_json(include_notebook=False)[source]

Convert Report to a JSON

class paperboy.config.report.ReportMetadataConfig(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Paperboy configuration object representing a Report (metadata component)

created

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

static from_json(jsn)[source]

Create ReportMetadata from a JSON

job

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

modified

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

notebook

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

output

A trait for unicode strings.

parameters

A trait for unicode strings.

run

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

strip_code

A boolean (True, False) trait.

template

A trait for unicode strings.

to_json(include_notebook=False)[source]

Convert ReportMetadata to a JSON

type

A trait for unicode strings.

userid

A trait for unicode strings.

username

A trait for unicode strings.

class paperboy.config.output.LocalOutputConfig(*args, **kwargs)[source]

Bases: paperboy.config.output.OutputConfig

Output report to local filesystem

clazz

alias of paperboy.output.local.LocalOutput

dir

A trait for unicode strings.

static from_json(jsn)[source]
to_json()[source]
type = 'local'
class paperboy.config.output.OutputConfig(*args, **kwargs)[source]

Bases: traitlets.traitlets.HasTraits

Base class for configuring output

type

A trait for unicode strings.

class paperboy.resources.logout.LogoutResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

Falcon resource for user authentication

auth_required = False
on_get(req, resp)[source]

Get logout page

on_post(req, resp)[source]

Log user out using authentication backend

class paperboy.resources.config.ConfigResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

Falcon resource to get form entries

on_get(req, resp)[source]

Get configuration page to create a new notebook/job/report

class paperboy.resources.register.RegisterResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

auth_required = False
on_get(req, resp)[source]

get registration page

on_post(req, resp)[source]

register a new user with storage backend

class paperboy.resources.job.JobDetailResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

Get details of specific job instance

class paperboy.resources.job.JobResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

List all job instances

on_post(req, resp)[source]

Create new or delete job instance

class paperboy.resources.html.HTMLResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

Falcon resource to service HTML files templated with jinja2

on_get(req, resp)[source]

Get templatized html and render

paperboy.resources.html.read(file)[source]

read a file from local disk

class paperboy.resources.static.StaticResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

Falcon resource to service CSS/JS/image/font files

auth_required = False
on_get(req, resp)[source]

read and return resource with appropriate mimetype

paperboy.resources.static.read(file)[source]

read static resource from disk (js/css/etc)

class paperboy.resources.scheduler.SchedulerResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

Get scheduler status of job and reports

class paperboy.resources.login.LoginResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

Falcon resource for user authentication

auth_required = False
on_get(req, resp)[source]

Get login page

on_post(req, resp)[source]

Log user in using authentication backend

class paperboy.resources.notebook.NotebookDetailResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

Get details of specific notebook instance

class paperboy.resources.notebook.NotebookResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

List all notebook instances

on_post(req, resp)[source]

Create new or delete old notebook instance

class paperboy.resources.autocomplete.AutocompleteResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

Autocompletion Falcon resource

on_get(req, resp)[source]

Get autocompletion.

This method calls the .search() method on the storage managers users/notebooks/jobs/reports attribute

class paperboy.resources.base.BaseResource(config, db=None, scheduler=None)[source]

Bases: object

Base falcon resource to handle shared attributes

class paperboy.resources.report.ReportDetailResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

Get details of specific report instance

class paperboy.resources.report.ReportResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

List all report instances

on_post(req, resp)[source]

Create new or delete old report instance

class paperboy.resources.status.StatusResource(*args, **kwargs)[source]

Bases: paperboy.resources.base.BaseResource

on_get(req, resp)[source]

get status report for notebooks/jobs/reports or all

class paperboy.output.sqla.SQLAOutput(config, *args, **kwargs)[source]

Bases: paperboy.output.base.BaseOutput

Output to sql database

write(report, output, *args, **kwargs)[source]

write report to output backend

class paperboy.output.local.LocalOutput(config, *args, **kwargs)[source]

Bases: paperboy.output.base.BaseOutput

Output to local filesystem

write(report, output, *args, **kwargs)[source]

write report to output backend

class paperboy.output.base.BaseOutput(config, *args, **kwargs)[source]

Bases: object

Abstract base class for Output types

write(report, *args, **kwargs)[source]

write report to output backend

class paperboy.scheduler.remote.RemoteScheduler(*args, **kwargs)[source]

Bases: paperboy.scheduler.base.BaseScheduler

Proxy methods to a remote worker instance

schedule(user, notebook, job, reports, *args, **kwargs)[source]

Schedule a job to generate reports for notebook and user :param user: user requesting the scheduling :type user: paperboy.config.User :param notebook: Notebook for the job :type notebook: paperboy.config.Notebook :param job: Job parameters :type job: paperboy.config.Job :param reports: Report configurations with parameters :type reports: [paperboy.config.Report]

status(user, params, session, *args, **kwargs)[source]

Get status for a given request :param req: :type req: falcon Request :param resp: :type resp: falcon response

unschedule(user, notebook, job, reports, *args, **kwargs)[source]

Unschedule a job to generate reports for notebook and user :param user: user requesting the scheduling :type user: paperboy.config.User :param notebook: Notebook for the job :type notebook: paperboy.config.Notebook :param job: Job parameters :type job: paperboy.config.Job :param reports: Report configurations with parameters :type reports: [paperboy.config.Report]

class paperboy.scheduler.luigi.luigi.LuigiScheduler(*args, **kwargs)[source]

Bases: paperboy.scheduler.base.BaseScheduler

luigi_command(path)[source]
static query()[source]

Get status of job/report tasks from luigi

schedule(user, notebook, job, reports, *args, **kwargs)[source]

Schedule a task for job composed of reports to be run on luigi

status(user, params, session, *args, **kwargs)[source]

Get status of job/report tasks

static template(config, user, notebook, job, reports, *args, **kwargs)[source]

jinja templatize luigi task for paperboy (paperboy.luigi.py)

unschedule(user, notebook, job, reports, *args, **kwargs)[source]

Remove the task for user and notebook composed of job running reports from luigi 2 parts, remove the task from disk and unschedule the task in cron

class paperboy.scheduler.local.local.LocalScheduler(*args, **kwargs)[source]

Bases: paperboy.scheduler.base.BaseScheduler

static query()[source]

Get status of job/reports

schedule(user, notebook, job, reports, *args, **kwargs)[source]

Schedule a DAG for job composed of reports to be run locally

status(user, params, session, *args, **kwargs)[source]

Get status of job/report tasks

unschedule(user, notebook, job, reports, *args, **kwargs)[source]

Remove the DAG for user and notebook composed of job running reports from local

class paperboy.scheduler.local.schedule.LocalProcessScheduler[source]

Bases: object

schedule(job, reports, job_dir, interval)[source]
unschedule(job_id)[source]
paperboy.scheduler.local.schedule.interval_to_schedule(cronjob)[source]

this function will be called roughly every minute and should parse the cronjob’s timing to see if the job should run in this minute

paperboy.scheduler.local.schedule.run_tasks(add_queue, delete_queue)[source]
class paperboy.scheduler.airflow.airflow.AirflowScheduler(*args, **kwargs)[source]

Bases: paperboy.scheduler.base.BaseScheduler

static query(engine)[source]

Get status of job/report DAGs from airflow’s database

schedule(user, notebook, job, reports, *args, **kwargs)[source]

Schedule a DAG for job composed of reports to be run on airflow

status(user, params, session, *args, **kwargs)[source]

Get status of job/report DAGs

static template(config, user, notebook, job, reports, *args, **kwargs)[source]

jinja templatize airflow DAG for paperboy (paperboy.airflow.py)

unschedule(user, notebook, job, reports, *args, **kwargs)[source]

Remove the DAG for user and notebook composed of job running reports from airflow 2 parts, remove the dag from disk and delete the dag from airflow’s database using the CLI

class paperboy.scheduler.base.BaseScheduler(config, db, *args, **kwargs)[source]

Bases: object

Scheduler abstract base class

schedule(user, notebook, job, reports, *args, **kwargs)[source]

Schedule a job to generate reports for notebook and user :param user: user requesting the scheduling :type user: paperboy.config.User :param notebook: Notebook for the job :type notebook: paperboy.config.Notebook :param job: Job parameters :type job: paperboy.config.Job :param reports: Report configurations with parameters :type reports: [paperboy.config.Report]

status(req, resp, *args, **kwargs)[source]

Get status for a given request :param req: :type req: falcon Request :param resp: :type resp: falcon response

unschedule(user, notebook, job, reports, *args, **kwargs)[source]

Unschedule a job to generate reports for notebook and user :param user: user requesting the scheduling :type user: paperboy.config.User :param notebook: Notebook for the job :type notebook: paperboy.config.Notebook :param job: Job parameters :type job: paperboy.config.Job :param reports: Report configurations with parameters :type reports: [paperboy.config.Report]

class paperboy.scheduler.base.DummyScheduler(config, db, *args, **kwargs)[source]

Bases: paperboy.scheduler.base.BaseScheduler

Dummy Scheduler class

static fakequery()[source]

If airflow not present, fake the results for now so the UI looks ok

schedule(user, notebook, job, reports, *args, **kwargs)[source]

Schedule a job to generate reports for notebook and user :param user: user requesting the scheduling :type user: paperboy.config.User :param notebook: Notebook for the job :type notebook: paperboy.config.Notebook :param job: Job parameters :type job: paperboy.config.Job :param reports: Report configurations with parameters :type reports: [paperboy.config.Report]

status(user, params, session, *args, **kwargs)[source]

Get status for a given request :param req: :type req: falcon Request :param resp: :type resp: falcon response

unschedule(user, notebook, job, reports, *args, **kwargs)[source]

Unschedule a job to generate reports for notebook and user :param user: user requesting the scheduling :type user: paperboy.config.User :param notebook: Notebook for the job :type notebook: paperboy.config.Notebook :param job: Job parameters :type job: paperboy.config.Job :param reports: Report configurations with parameters :type reports: [paperboy.config.Report]

paperboy.scheduler.base.interval_to_cron(interval, start_time)[source]
paperboy.scheduler.cron.schedule_cron(command, interval, crontab=None)[source]
paperboy.scheduler.cron.unschedule_cron(command, crontab=None)[source]