rework documentation

docs/advanced_usage.rst

@@ -0,0 +1,629 @@
+Advanced usage
+==============
+
+`Burp-UI`_ has been written with modularity in mind. The aim is to support
+`Burp`_ from the stable to the latest versions. `Burp`_ exists in two major
+versions: 1.x.x and 2.x.x.
+
+Both `Versions`_ are supported by `Burp-UI`_ thanks to its modular design.
+The consequence is you have various options in the configuration file to suite
+everybody needs.
+
+There are also different modules to support `Authentication`_ and `ACL`_ within
+the web-interface.
+
+.. warning::
+    `Burp-UI`_ tries to be as less intrusive as possible, nevertheless it ships
+    with the ability to manage `Burp`_'s configuration files.
+    This feature **requires** `Burp-UI`_ to be launched on the **same** server
+    that hosts your `Burp`_ instance.
+    You also have to make sure the user that runs `Burp-UI`_ has **enough**
+    privileges to edit those files.
+
+
+Configuration
+-------------
+
+The `burpui.cfg`_ configuration file contains a ``[Global]`` section as follow:
+
+::
+
+    [Global]
+    # burp server version 1 or 2
+    version = 2
+    # Handle multiple bui-servers or not
+    # If set to 'false', you will need to declare at least one 'Agent' section (see
+    # bellow)
+    single = true
+    # authentication plugin (mandatory)
+    # list the misc/auth directory to see the available backends
+    # to disable authentication you can set "auth = none"
+    # you can also chain multiple backends. Example: "auth = ldap,basic"
+    # the order will be respected unless you manually set a higher backend priority
+    auth = basic
+    # acl plugin
+    # list misc/acl directory to see the available backends
+    # default is no ACL
+    acl = basic
+    # You can change the prefix if you are behind a reverse-proxy under a custom
+    # root path. For example: /burpui
+    # You can also configure your reverse-proxy to announce the prefix through the
+    # 'X-Script-Name' header. In this case, the bellow prefix will be ignored in
+    # favour of the one announced by your reverse-proxy
+    prefix = none
+
+
+Each option is commented, but here is a more detailed documentation:
+
+- *version*: What version of `Burp`_ this `Burp-UI`_ instance manages. Can
+  either be *1* or *2*. This parameter determines which backend is loaded at
+  runtime.
+
+  (see `Versions`_ for more details)
+- *single*: `Burp-UI`_ can run in two different modes. If it runs in
+  single mode (meaning you set this parameter to *true*), you can only
+  address **one** `Burp`_ server of the version specified by the previous
+  parameter.
+
+  If this option is set to *false*, `Burp-UI`_ will run as a *proxy* allowing
+  you to address multiple `Burp`_ servers. In this mode, you need to configure
+  **at least one** *Agent* section in your configuration file. You also need to
+  run one ``bui-agent`` per server.
+
+  (see `Modes`_ for more details)
+- *auth*: What `Authentication`_ backend to use.
+- *acl*: What `ACL`_ module to use.
+- *prefix*: You can host `Burp-UI`_ behind a sub-root path. See the `gunicorn
+  <gunicorn.html#sub-root-path>`__ page for details.
+
+
+There is also a ``[UI]`` section in which you can configure some *UI*
+parameters:
+
+::
+
+    [UI]
+    # refresh interval of the pages in seconds
+    refresh = 180
+    # refresh interval of the live-monitoring page in seconds
+    liverefresh = 5
+
+
+Each option is commented, but here is a more detailed documentation:
+
+- *refresh*: Time in seconds between two refresh of the interface.
+- *liverefresh*: Time in seconds between two refresh of the *live-monitor* page.
+
+Production
+----------
+
+The `burpui.cfg`_ configuration file contains a ``[Production]`` section as
+follow:
+
+::
+
+    [Production]
+    # storage backend for session and cache
+    # may be either 'default' or 'redis'
+    storage = default
+    # session database to use
+    # may also be a backend url like: redis://localhost:6379/0
+    # if set to 'redis', the backend url defaults to:
+    # redis://<redis_host>:<redis_port>/0
+    # where <redis_host> is the host part, and <redis_port> is the port part of
+    # the below "redis" setting
+    session = default
+    # cache database to use
+    # may also be a backend url like: redis://localhost:6379/0
+    # if set to 'redis', the backend url defaults to:
+    # redis://<redis_host>:<redis_port>/1
+    # where <redis_host> is the host part, and <redis_port> is the port part of
+    # the below "redis" setting
+    cache = default
+    # redis server to connect to
+    redis = localhost:6379
+    # whether to use celery or not
+    # may also be a broker url like: redis://localhost:6379/0
+    # if set to "true", the broker url defaults to:
+    # redis://<redis_host>:<redis_port>/2
+    # where <redis_host> is the host part, and <redis_port> is the port part of
+    # the above "redis" setting
+    celery = false
+    # database url to store some persistent data
+    # none or a connect string supported by SQLAlchemy:
+    # http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls
+    # example: sqlite:////var/lib/burpui/store.db
+    database = none
+    # whether to rate limit the API or not
+    # may also be a redis url like: redis://localhost:6379/0
+    # if set to "true" or "redis" or "default", the url defaults to:
+    # redis://<redis_host>:<redis_port>/3
+    # where <redis_host> is the host part, and <redis_port> is the port part of
+    # the above "redis" setting
+    # Note: the limiter only applies to the API routes
+    limiter = false
+    # limiter ratio
+    # see https://flask-limiter.readthedocs.io/en/stable/#ratelimit-string
+    ratio = 20/minute
+
+
+Experimental
+------------
+
+There is a ``[Experimental]`` section for features that have not been deeply
+tested:
+
+::
+
+    [Experimental]
+    ## This section contains some experimental features that have not been deeply
+    ## tested yet
+    # enable zip64 feature. Python doc says:
+    # « ZIP64 extensions are disabled by default because the default zip and unzip
+    # commands on Unix (the InfoZIP utilities) don’t support these extensions. »
+    zip64 = false
+
+
+These options are also available in the `bui-agent`_ configuration file.
+
+Security
+--------
+
+The ``[Security]`` section contains options to harden the security of the
+application:
+
+::
+
+    [Security]
+    ## This section contains some security options. Make sure you understand the
+    ## security implications before changing these.
+    # list of 'root' paths allowed when sourcing files in the configuration.
+    # Set this to 'none' if you don't want any restrictions, keeping in mind this
+    # can lead to accessing sensible files. Defaults to '/etc/burp'.
+    # Note: you can have several paths separated by comas.
+    # Example: /etc/burp,/etc/burp.d
+    includes = /etc/burp
+    # if files already included in config do not respect the above restriction, we
+    # prune them
+    enforce = false
+    # enable certificates revocation
+    revoke = false
+    # remember_cookie duration in days
+    cookietime = 14
+    # whether to use a secure cookie for https or not. If set to false, cookies
+    # won't have the 'secure' flag.
+    # This setting is only useful when HTTPS is detected
+    scookie = true
+    # application secret to secure cookies. If you don't set anything, the default
+    # value is 'random' which will generate a new secret after every restart of your
+    # application. You can also set it to 'none' although this is not recommended.
+    appsecret = random
+
+
+Some of these options are also available in the `bui-agent`_ configuration file.
+
+Modes
+-----
+
+`Burp-UI`_ provides two modes:
+
+- `Single`_
+- `Multi-Agent`_
+
+These modes allow you to either access a single `Burp`_ server or multiple
+`Burp`_ servers hosted on separated hosts.
+
+
+Single
+^^^^^^
+
+This mode is the **default** and the easiest one. It can be activated by setting
+the *single* parameter in the ``[Global]`` section of your `burpui.cfg`_
+file to *true*:
+
+::
+
+    [Global]
+    single = true
+
+
+That's all you need to do for this mode to work.
+
+
+Multi-Agent
+^^^^^^^^^^^
+
+This mode allows you access multiple `Burp`_ servers through the `bui-agent`_.
+The architecture is available on the bui-agent
+`page <buiagent.html#architecture>`__.
+
+
+To enable this mode, you need to set the *single* parameter of the
+``[Global]`` section of your `burpui.cfg`_ file to *false*:
+
+::
+
+    [Global]
+    single = false
+
+
+Once this mode is enabled, you have to create **one** ``[Agent]`` section
+**per** agent you want to connect to in your `burpui.cfg`_ file:
+
+::
+
+    # If you set single to 'false', add at least one section like this per
+    # bui-agent
+    [Agent:agent1]
+    # bui-agent address
+    host = 192.168.1.1
+    # bui-agent port
+    port = 10000
+    # bui-agent password
+    password = azerty
+    # enable SSL
+    ssl = true
+
+    [Agent:agent2]
+    # bui-agent address
+    host = 192.168.2.1
+    # bui-agent port
+    port = 10000
+    # bui-agent password
+    password = ytreza
+    # enable SSL
+    ssl = true
+
+
+.. note:: The sections must be called ``[Agent:<label>]`` (case sensitive)
+
+To configure your agents, please refer to the `bui-agent`_ page.
+
+
+Versions
+--------
+
+`Burp-UI`_ ships with two different backends:
+
+- `Burp1`_
+- `Burp2`_
+
+These backends allow you to either connect to a `Burp`_ server version 1.x.x or
+2.x.x.
+
+.. note::
+    If you are using a `Burp`_ server version 2.x.x you **have** to use the
+    `Burp2`_ backend, no matter what `Burp`_'s protocol you are using.
+
+
+Burp1
+^^^^^
+
+.. note::
+    Make sure you have read and understood the `requirements
+    <requirements.html#burp1>`__ first.
+
+The *burp-1* backend can be enabled by setting the *version* option to *1* in
+the ``[Global]`` section of your `burpui.cfg`_ file:
+
+::
+
+    [Global]
+    version = 1
+
+
+Now you can add *burp-1* backend specific options:
+
+::
+
+    # burp1 backend specific options
+    [Burp1]
+    # burp status address (can only be '127.0.0.1' or '::1')
+    bhost = ::1
+    # burp status port
+    bport = 4972
+    # burp binary
+    burpbin = /usr/sbin/burp
+    # vss_strip binary
+    stripbin = /usr/sbin/vss_strip
+    # burp client configuration file used for the restoration (Default: None)
+    bconfcli = /etc/burp/burp.conf
+    # burp server configuration file used for the setting page
+    bconfsrv = /etc/burp/burp-server.conf
+    # temporary directory to use for restoration
+    tmpdir = /tmp
+
+
+Each option is commented, but here is a more detailed documentation:
+
+- *bhost*: The address of the `Burp`_ server. In burp-1.x.x, it can only be
+  *127.0.0.1* or *::1*
+- *bport*: The port of `Burp`_'s status port.
+- *burpbin*: Path to the `Burp`_ binary (used for restorations).
+- *stripbin*: Path to the `Burp`_ *vss_strip* binary (used for restorations).
+- *bconfcli*: Path to the `Burp`_ client configuration file (see 
+  `restoration <installation.html#restoration>`__).
+- *bconfsrv*: Path to the `Burp`_ server configuration file.
+- *tmpdir*: Path to a temporary directory where to perform restorations.
+
+
+Burp2
+^^^^^
+
+.. note::
+    Make sure you have read and understood the `requirements
+    <requirements.html#burp2>`__ first.
+
+.. note::
+    The `gunicorn <gunicorn.html#daemon>`__ documentation may help you
+    configuring your system.
+
+The *burp-2* backend can be enabled by setting the *version* option to *2* in
+the ``[Global]`` section of your `burpui.cfg`_ file:
+
+::
+
+    [Global]
+    version = 2
+
+
+Now you can add *burp-2* backend specific options:
+
+::
+
+    # burp2 backend specific options
+    [Burp2]
+    # burp binary
+    burpbin = /usr/sbin/burp
+    # vss_strip binary
+    stripbin = /usr/bin/vss_strip
+    # burp client configuration file used for the restoration (Default: None)
+    bconfcli = /etc/burp/burp.conf
+    # burp server configuration file used for the setting page
+    bconfsrv = /etc/burp/burp-server.conf
+    # temporary directory to use for restoration
+    tmpdir = /tmp
+    # how many time to wait for the monitor to answer (in seconds)
+    timeout = 5
+
+
+Each option is commented, but here is a more detailed documentation:
+
+- *burpbin*: Path to the `Burp`_ binary (used for restorations).
+- *stripbin*: Path to the `Burp`_ *vss_strip* binary (used for restorations).
+- *bconfcli*: Path to the `Burp`_ client configuration file (see
+  `restoration <installation.html#restoration>`__).
+- *bconfsrv*: Path to the `Burp`_ server configuration file.
+- *tmpdir*: Path to a temporary directory where to perform restorations.
+- *timeout*: Time to wait for the monitor to answer in seconds.
+
+
+Authentication
+--------------
+
+`Burp-UI`_ provides some authentication backends in order to restrict access
+only to granted users.
+There are currently three different backends:
+
+- `LDAP`_
+- `Basic`_
+- `Local`_
+
+To disable the *authentication* backend, set the *auth* option of the
+``[Global]`` section of your `burpui.cfg`_ file to *none*:
+
+::
+
+    [Global]
+    auth = none
+
+
+You can use multiple backends, they will be sorted by priority or in the order
+they are defined if no priority is found.
+If a user is present in several backends, the first one that matches both login
+and password will be used.
+
+Example:
+
+::
+
+    [Global]
+    auth = basic,ldap
+
+
+LDAP
+^^^^
+
+The *ldap* authentication backend has some dependencies, please refer to the
+`requirements <requirements.html#ldap>`_ page. To enable this backend, you need
+to set the *auth* option of the ``[Global]`` section of your `burpui.cfg`_ file
+to *ldap*:
+
+::
+
+    [Global]
+    auth = ldap
+
+
+Now you can add *ldap* specific options:
+
+::
+
+    # ldapauth specific options
+    [LDAP]
+    # Backend priority. Higher is first
+    priority = 1
+    # LDAP host
+    host = 127.0.0.1
+    # LDAP port
+    port = 389
+    # Encryption type to LDAP server (none, ssl or tls)
+    # - try tls if unsure, otherwise ssl on port 636
+    encryption = tls
+    # specifies if the server certificate must be validated, values can be:
+    #  - none (certificates are ignored)
+    #  - optional (not required, but validated if provided)
+    #  - required (required and validated)
+    validate = none
+    # SSL or TLS version to use, can be one of the following:
+    #  - SSLv2
+    #  - SSLv3
+    #  - SSLv23
+    #  - TLSv1
+    #  - TLSv1_1 (Available only with openssl version 1.0.1+, requires python 2.7.9 or higher)
+    version = TLSv1
+    # the file containing the certificates of the certification authorities
+    cafile = none
+    # Attribute to use when searching the LDAP repository
+    #searchattr = sAMAccountName
+    searchattr = uid
+    # LDAP filter to find users in the LDAP repository
+    #  - {0} will be replaced by the search attribute
+    #  - {1} will be replaced by the login name
+    filter = (&({0}={1})(burpui=1))
+    #filter = (&({0}={1})(|(userAccountControl=512)(userAccountControl=66048)))
+    # LDAP base
+    base = "ou=users,dc=example,dc=com"
+    # Binddn to list existing users
+    binddn = "cn=admin,dc=example,dc=com"
+    # Bindpw to list existing users
+    bindpw = Sup3rS3cr3tPa$$w0rd
+
+
+.. note:: The *host* options accepts URI style (ex: ldap://127.0.0.1:389)
+
+.. warning:: The quotes (") around *base* and *binddn* are **MANDATORY**
+
+Basic
+^^^^^
+
+In order for the *basic* authentication backend to be enabled, you need to set
+the *auth* option of the ``[Global]`` section of your `burpui.cfg`_ file to
+*basic*:
+
+::
+
+    [Global]
+    auth = basic
+
+
+Now you can add *basic* specific options:
+
+::
+
+    # basicauth specific options
+    # Note: in case you leave this section commented, the default login/password
+    # is admin/admin
+    [BASIC]
+    # Backend priority. Higher is first
+    priority = 2
+    admin = pbkdf2:sha1:1000$12345678$password
+    user1 = pbkdf2:sha1:1000$87654321$otherpassword
+
+
+.. note::
+    Each line defines a new user with the *key* as the username and the *value*
+    as the password
+
+.. warning::
+    Since v0.3.0, passwords must be hashed (see `manage <manage.html#users>`_ to
+    know how to create new users with hashed passwords)
+
+Local
+^^^^^
+
+In order for the *local* authentication backend to be enabled, you need to set
+the *auth* option of the ``[Global]`` section of your `burpui.cfg`_ file to
+*local*:
+
+::
+
+    [Global]
+    auth = local
+
+
+Now you can add *local* specific options:
+
+::
+
+    # localauth specific options
+    # Note: if not running as root, then burp-ui must be run as group 'shadow' to
+    # allow PAM to work
+    [LOCAL]
+    # Backend priority. Higher is first
+    priority = 3
+    # List of local users allowed to login. If you don't set this setting, every
+    # local user will be able to login
+    users = user1,user2
+
+
+ACL
+---
+
+`Burp-UI`_ implements some mechanisms to restrict access on some resources only
+for some users.
+There is currently only one backend:
+
+- `Basic ACL`_
+
+To disable the *acl* backend, set the *acl* option of the ``[Global]`` section
+of your `burpui.cfg`_ file to *none*:
+
+::
+
+    [Global]
+    acl = none
+
+
+Basic ACL
+^^^^^^^^^
+
+
+The *basic* acl backend can be enabled by setting the *acl* option of the
+``[Global]`` section of your `burpui.cfg`_ file to *basic*:
+
+::
+
+    [Global]
+    acl = basic
+
+
+Now you can add *basic acl* specific options:
+
+::
+
+    # basicacl specific options
+    # Note: in case you leave this section commented, the user 'admin' will have
+    # access to all clients whereas other users will only see the client that have
+    # the same name
+    [BASIC:ACL]
+    admin = user1,user2
+    # Please note the double-quotes and single-quotes on the following lines are
+    # mandatory!
+    # You can also overwrite the default behavior by specifying which clients a
+    # user can access
+    user3 = '["client4", "client5"]'
+    # In case you are not in a single mode, you can also specify which clients
+    # a user can access on a specific Agent
+    user4 = '{"agent1": ["client6", "client7"], "agent2": ["client8"]}'
+
+
+.. warning:: The double-quotes and single-quotes are **MANDATORY**
+
+
+By default, if a user is named ``admin`` it will be granted the admin role.
+Here are the default grants:
+
+
+1. admin => you can do anything
+2. non admin => you can only see the client that matches your username
+3. custom => you can manually assign username to clients using the syntax
+   ``username = client1,client2`` or
+   ``username = '{"agent1": ["client1-1"], "agent2": ["client2-3", "client2-4"]}'``
+   (if you are running a multi-agent setup)
+
+
+.. _Burp: http://burp.grke.org/
+.. _Burp-UI: https://git.ziirish.me/ziirish/burp-ui
+.. _burpui.cfg: https://git.ziirish.me/ziirish/burp-ui/blob/master/share/burpui/etc/burpui.sample.cfg
+.. _bui-agent: buiagent.html

docs/api.rst

@@ -4,6 +4,18 @@ API
 Here are the different routes provided by the application. You can implement
 whatever front-end you like on top of it.
 
+The API supports HTTP Basic authentication through the *Authorization* Header.
+By default, each authorization is only valid for **one** request (ie. the
+sessions are automatically revoked after each request complete). You can ask for
+reusable sessions though.
+Here are the HTTP headers supported:
+
+- ``X-Reuse-Session``: set it to ``true`` to be able to reuse sessions
+- ``X-Language``: set it to whatever supported language you want
+
+Don't forget to call ``/logout`` once you're done if you choose to use reusable
+sessions.
+
 .. autoflask:: burpui._rtfd:app
         :blueprints: api
         :undoc-static:

docs/basic_usage.rst

@@ -0,0 +1,125 @@
+Basic usage
+===========
+
+General Instructions
+--------------------
+
+Restoration
+^^^^^^^^^^^
+
+`Burp-UI`_ tries to be as less intrusive as possible with `Burp`_ internals.
+In order to make the *online* restoration/download functionality work, you
+need to check a few things:
+
+1. Provide the full path of the burp (client) binary file (field *burpbin* in 
+   `burp-ui configuration <advanced_usage.html#versions>`__)
+2. Provide a burp-client configuration file (field *bconfcli* in
+   `burp-ui configuration <advanced_usage.html#versions>`__)
+3. Provide the full path of an empty directory where a temporary restoration
+   will be made. This involves you have enough space left on that location on
+   the server that runs `Burp-UI`_
+4. Launch `Burp-UI`_ with a user that can proceed restorations and that can
+   write in the directory mentioned above
+5. Make sure the client provided in 2. can restore files of other clients
+   (option *restore_client* in burp-server configuration).
+   The *restore_client* is the *cname* you provided in your client configuration
+   file (see 2.)
+
+Burp 2
+^^^^^^
+
+When using the `burp2 backend <usage.html#burp2>`_, `Burp-UI`_ can be executed
+on any machine as long as you can access the burp status port, but you will not
+be able to edit the burp server configuration file within the *settings* view of
+`Burp-UI`_.
+You also need to configure a *restore_client* on your burp server corresponding
+to the client you will use through `Burp-UI`_ (see the `restoration procedure
+<#restoration>`_ above for details on how to do this)
+
+
+Options
+-------
+
+::
+
+    usage: burp-ui [-h] [-v] [-d] [-V] [-c <CONFIG>] [-l <FILE>]
+                   [-i <MIGRATIONSDIR>]
+                   ...
+
+    positional arguments:
+      remaining
+
+    optional arguments:
+      -h, --help            show this help message and exit
+      -v, --verbose         increase output verbosity (e.g., -vv is more verbose
+                            than -v)
+      -d, --debug           enable debug mode
+      -V, --version         print version and exit
+      -c <CONFIG>, --config <CONFIG>
+                            burp-ui configuration file
+      -l <FILE>, --logfile <FILE>
+                            output logs in defined file
+      -i <MIGRATIONSDIR>, --migrations <MIGRATIONSDIR>
+                            migrations directory
+
+
+.. note:: Since v0.4.0, the *bind* and *port* options are not supported anymore.
+          You can still force the port and listening address as explained bellow
+          or you can have a look at the `upgrading <upgrading.html#v0.4.0>`_
+          section of the documentation for further details.
+
+Developer options
+-----------------
+
+Since *v0.4.0*, `Burp-UI`_ uses the new Flask's *CLI* module. This change brings
+new options to help you debug your development environment:
+
+::
+
+    Usage: flask run [OPTIONS]
+
+      Runs a local development server for the Flask application.
+
+      This local server is recommended for development purposes only but it can
+      also be used for simple intranet deployments.  By default it will not
+      support any sort of concurrency at all to simplify debugging.  This can be
+      changed with the --with-threads option which will enable basic
+      multithreading.
+
+      The reloader and debugger are by default enabled if the debug flag of
+      Flask is enabled and disabled otherwise.
+
+    Options:
+      -h, --host TEXT                 The interface to bind to.
+      -p, --port INTEGER              The port to bind to.
+      --reload / --no-reload          Enable or disable the reloader.  By default
+                                      the reloader is active if debug is enabled.
+      --debugger / --no-debugger      Enable or disable the debugger.  By default
+                                      the debugger is active if debug is enabled.
+      --eager-loading / --lazy-loader
+                                      Enable or disable eager loading.  By default
+                                      eager loading is enabled if the reloader is
+                                      disabled.
+      --with-threads / --without-threads
+                                      Enable or disable multithreading.
+      --help                          Show this message and exit.
+
+
+Some options are redundant with `Burp-UI`_'s one.
+
+Example
+-------
+
+By default, the embedded server listens on *127.0.0.1* on port *5000*, if you
+wish to change this, you would run something like:
+
+::
+
+    burp-ui -- -h 0.0.0.0 -p 8080
+
+
+.. note:: The ``--`` is mandatory.
+
+
+.. _Burp-UI: https://git.ziirish.me/ziirish/burp-ui
+.. _Burp: http://burp.grke.net/

docs/buiagent.rst

@@ -121,12 +121,12 @@ Each option is commented, but here is a more detailed documentation:
 - *sslcert*: What SSL certificate to use when SSL is enabled.
 - *sslkey*: What SSL key to use when SSL is enabled.
 - *version*: What version of `Burp`_ this `bui-agent`_ instance manages. (see
-  `Burp-UI versions <usage.html#versions>`__ for more details)
+  `Burp-UI versions <advanced_usage.html#versions>`__ for more details)
 - *password*: The shared secret between the `Burp-UI`_ server and `bui-agent`_.
 
 As with `Burp-UI`_, you need a specific section depending on the *version*
-value. Please refer to the `Burp-UI versions <usage.html#versions>`__ section
+value. Please refer to the `Burp-UI versions <advanced_usage.html#versions>`__
-for more details.
+section for more details.
 
 Daemon
 ------

docs/celery.rst

@@ -3,7 +3,7 @@ Celery
 
 Since *v0.3.0*, `Burp-UI`_ supports asynchronous operations thanks to `Celery`_.
 In order to use this feature, you need to enable it in the configuration (see
-`Production <usage.html#production>`__ section)
+`Production <advanced_usage.html#production>`__ section)
 
 You will also need some extra requirements:
 

docs/docker.rst

@@ -29,7 +29,7 @@ provided. There are a few variables supported to setup your system:
  - **BURP_SERVER_CONFIG** - Specify the path of the burp-server configuration
    file. It defaults to "/etc/burp/burp-server.conf".
  - **DATABASE_URL** - Specify the URL of the database to connect to. It defaults
-   to "sqlite:////var/lib/burpui/store.db".
+   to "mysql://burpui:burpui@mariadb/burpuidb".
  - **REDIS_SERVER** - Specify the address of the redis server. It defaults to
    "redis:6379".
  - **BURP_SERVER_ADDR** - Specify the address of the burp-server status port.

docs/faq.rst

@@ -13,8 +13,8 @@ How to start using Burp-UI?
 ---------------------------
 
 You may find all the basic informations to get started with `Burp-UI`_ in the
-`README`_ file. You can also read the `step-by-step <step-by-step.html>`_ page
+`README`_ file and in this documentation. You can also read the
-to get started.
+`step-by-step <step-by-step.html>`_ page for some detailed use-cases.
 
 How does Burp-UI work?
 ----------------------
@@ -22,8 +22,8 @@ How does Burp-UI work?
 The answer depends whether you are using burp 1.x or burp 2.x. Basically,
 `Burp-UI`_ tries to provide a consistent API between the *Frontend* (the UI) and
 the burp server. To do so, it implements two *Backends*: burp-1 and burp-2.
-You can select either of these with the `version <usage.html#versions>`__ flag
+You can select either of these with the `version <advanced_usage.html#versions>`__
-in your configuration.
+flag in your configuration.
 
 You can also refer to the `Architecture <architecture.html>`__ page of the
 documentation to know more about those backends.
@@ -31,7 +31,7 @@ documentation to know more about those backends.
 How to configure my *firewall*?
 -------------------------------
 
-When running `Burp-UI`_ in standalone `mode <usage.html#versions>`__, the
+When running `Burp-UI`_ in single `mode <advanced_usage.html#versions>`__, the
 embedded webserver listens on port **5000** on all interfaces.
 
 The `Burp-UI`_ agents listen on port **10000** by default.
@@ -42,7 +42,7 @@ What are the default credentials?
 ---------------------------------
 
 The default login / password is *admin* / *admin* with the
-`basic <usage.html#basic>`__ authentication backend.
+`basic <advanced_usage.html#basic>`__ authentication backend.
 
 How does the online restoration feature work?
 ---------------------------------------------
@@ -62,8 +62,8 @@ Because of this workflow, and especially the use of the ``-C`` flag you need to
 tell your burp-server the client used by `Burp-UI`_ can perform a restoration
 for a different client.
 You can refer to the `restoration <installation.html#restoration>`__ section of
-this documentation along with the `version <usage.html#versions>`__ section for
+this documentation along with the `version <advanced_usage.html#versions>`__
-more details.
+section for more details.
 
 What does the server-initiated restoration feature do and how to make it work?
 ------------------------------------------------------------------------------
@@ -125,10 +125,19 @@ Why using SQL?
 The SQL database is currently used to keep a track of several meta-data.
 Again, it is totally optional to use it.
 
+Burp-UI does not seem to understand the *bind* and *port* options anymore, what should I do?
+--------------------------------------------------------------------------------------------
+
+Since v0.4.0, the new Flask development server is used when running in
+*single* mode. The *bind* and *port* options are not read anymore.
+You can either run `Burp-UI`_ with the ``-- -h x.x.x.x -p yyyy`` flags or use
+the legacy launcher ``python -m burpui -m legacy [--help]``.
+See the `upgrading <upgrading.html#v0-4-0>`__ page for details.
+
 Burp-UI does not work anymore since I upgraded it, what can I do?
 -----------------------------------------------------------------
 
-Make sure you read the `upgrading <upgrading.html>`_ page in case some breaking
+Make sure you read the `upgrading <upgrading.html>`__ page in case some breaking
 changes occurred.
 
 I am getting errors while restoring large files (>3GB), what should I do?

docs/gunicorn.rst

@@ -169,8 +169,9 @@ There is a sample configuration file available
 Systemd
 ^^^^^^^
 
-On non debian systems, the handler script may not be available. You will then
+If you are not running on debian or you prefer not to use the gunicorn debian
-have to create your own service. We can do this for systemd for example:
+package, the handler script may not be available. You will then have to create
+your own service. We can do this for systemd for example:
 
 ::
 
@@ -234,7 +235,7 @@ Sub-root path
 You can host `Burp-UI`_ behind a sub-root path. For instance ``/burpui``.
 To accomplish this, you can either setup your reverse-proxy to announce the
 desired *prefix*, or you can use the ``prefix`` option in your `Burp-UI`_
-configuration file (see `usage <usage.html>`_ for details).
+configuration file (see `usage <advanced_usage.html>`_ for details).
 
 If you want to configure this reverse-proxy side, you need to announce the HTTP
 Header ``X-Script-Name``.
@@ -293,8 +294,8 @@ It is using `Gunicorn`_ along with Nginx as described above.
 In order to improve performances, `Redis`_ can be used to cache sessions and
 various API calls.
 
-See the `production <usage.html#production>`_ section of the
+See the `production <advanced_usage.html#production>`_ section of the
-`usage <usage.html>`_ page.
+`usage <advanced_usage.html>`_ page.
 
 .. _Gunicorn: http://gunicorn.org/
 .. _Burp-UI: https://git.ziirish.me/ziirish/burp-ui

docs/index.rst

@@ -21,14 +21,16 @@ Documentation
    introduction
    requirements
    installation
+   basic_usage
+   running
    upgrading
-   architecture
+   advanced_usage
-   usage
    manage
    celery
    gunicorn
    docker
    buiagent
+   architecture
    contributing
    changelog
    faq

docs/installation.rst

@@ -56,126 +56,6 @@ scratch or to upgrade your current version to the latest one:
     easy_install --upgrade burp-ui
 
 
-General Instructions
---------------------
-
-Restoration
-^^^^^^^^^^^
-
-`Burp-UI`_ tries to be as less intrusive as possible with `Burp`_ internals.
-In order to make the *online* restoration/download functionality work, you
-need to check a few things:
-
-1. Provide the full path of the burp (client) binary file (field *burpbin* in 
-   `burp-ui configuration <usage.html#versions>`__)
-2. Provide a burp-client configuration file (field *bconfcli* in
-   `burp-ui configuration <usage.html#versions>`__)
-3. Provide the full path of an empty directory where a temporary restoration
-   will be made. This involves you have enough space left on that location on
-   the server that runs `Burp-UI`_
-4. Launch `Burp-UI`_ with a user that can proceed restorations and that can
-   write in the directory mentioned above
-5. Make sure the client provided in 2. can restore files of other clients
-   (option *restore_client* in burp-server configuration).
-   The *restore_client* is the *cname* you provided in your client configuration
-   file (see 2.)
-
-Burp 2
-^^^^^^
-
-When using the `burp2 backend <usage.html#burp2>`_, `Burp-UI`_ can be executed
-on any machine as long as you can access the burp status port, but you will not
-be able to edit the burp server configuration file within the *settings* view of
-`Burp-UI`_.
-You also need to configure a *restore_client* on your burp server corresponding
-to the client you will use through `Burp-UI`_ (see the `restoration procedure
-<#restoration>`_ above)
-
-
-Options
--------
-
-::
-
-    usage: burp-ui [-h] [-v] [-d] [-V] [-c <CONFIG>] [-l <FILE>]
-                   [-i <MIGRATIONSDIR>]
-                   ...
-
-    positional arguments:
-      remaining
-
-    optional arguments:
-      -h, --help            show this help message and exit
-      -v, --verbose         increase output verbosity (e.g., -vv is more verbose
-                            than -v)
-      -d, --debug           enable debug mode
-      -V, --version         print version and exit
-      -c <CONFIG>, --config <CONFIG>
-                            burp-ui configuration file
-      -l <FILE>, --logfile <FILE>
-                            output logs in defined file
-      -i <MIGRATIONSDIR>, --migrations <MIGRATIONSDIR>
-                            migrations directory
-
-
-.. note:: Since v0.4.0, the *bind* and *port* options are not supported anymore.
-          You can still force the port and listening address as explained bellow
-          or you can have a look at the `upgrading <upgrading.html#v0.4.0>`_
-          section of the documentation for further details.
-
-Developer options
------------------
-
-Since *v0.4.0*, `Burp-UI`_ uses the new Flask's *CLI* module. This change brings
-new options to help you debug your development environment:
-
-::
-
-    Usage: flask run [OPTIONS]
-
-      Runs a local development server for the Flask application.
-
-      This local server is recommended for development purposes only but it can
-      also be used for simple intranet deployments.  By default it will not
-      support any sort of concurrency at all to simplify debugging.  This can be
-      changed with the --with-threads option which will enable basic
-      multithreading.
-
-      The reloader and debugger are by default enabled if the debug flag of
-      Flask is enabled and disabled otherwise.
-
-    Options:
-      -h, --host TEXT                 The interface to bind to.
-      -p, --port INTEGER              The port to bind to.
-      --reload / --no-reload          Enable or disable the reloader.  By default
-                                      the reloader is active if debug is enabled.
-      --debugger / --no-debugger      Enable or disable the debugger.  By default
-                                      the debugger is active if debug is enabled.
-      --eager-loading / --lazy-loader
-                                      Enable or disable eager loading.  By default
-                                      eager loading is enabled if the reloader is
-                                      disabled.
-      --with-threads / --without-threads
-                                      Enable or disable multithreading.
-      --help                          Show this message and exit.
-
-
-Some options are redundant with `Burp-UI`_'s one.
-
-Example
--------
-
-By default, the embedded server listens on *127.0.0.1* on port *5000*, if you
-wish to change this, you would run something like:
-
-::
-
-    burp-ui -- -h 0.0.0.0 -p 8080
-
-
-.. note:: The ``--`` is mandatory.
-
-
 .. _Flask: http://flask.pocoo.org/
 .. _burpui.cfg: https://git.ziirish.me/ziirish/burp-ui/blob/master/share/burpui/etc/burpui.sample.cfg
 .. _Burp-UI: https://git.ziirish.me/ziirish/burp-ui

docs/introduction.rst

@@ -1,11 +1,11 @@
 Introduction
 ============
 
-`Burp-UI`_ is a web-based interface for `Burp`_. It's purpose is to give you a
+`Burp-UI`_ is a web-based interface for `Burp`_. Its purpose is to give you a
 *nice* way to monitor your backups with some dashboards, but you will also have
 the ability to download files from backups and to configure your burp-server.
 
-The project also provides a full documented `API <api.html>`_ so that you can
+The project also provides a fully documented `API <api.html>`_ so that you can
 develop any front-end you like on top of it. The core will take care of the
 communication with the burp server(s) for you.
 

docs/manage.rst

@@ -73,7 +73,8 @@ Database
 --------
 
 To manage database migration, you first need to enable database support within
-your configuration file (see `Production <usage.html#production>`__ section)
+your configuration file (see `Production <advanced_usage.html#production>`__
+section)
 
 You will also need some extra requirements:
 
@@ -116,7 +117,7 @@ You can create new users using the ``bui-manage`` file like this:
     bui-manage create_user <new_username>
 
 
-By default, the script will create new users for the `Basic <usage.html#basic>`_
+By default, the script will create new users for the `Basic <advanced_usage.html#basic>`_
 authentication backend.
 Without further details, a new password will be generated.
 You can either provide a password through the command line or tell the script to

docs/requirements.rst

@@ -93,8 +93,8 @@ If you want to rate-limit the API, you will need additional dependencies too:
 Burp1
 -----
 
-The `burp1 backend <usage.html#burp1>`__ supports burp versions from 1.3.48 to
+The `burp1 backend <advanced_usage.html#burp1>`__ supports burp versions from
-1.4.40.
+1.3.48 to 1.4.40.
 With these versions of burp, the status port is only listening on the local
 machine loopback interface (ie. ``localhost`` or ``127.0.0.1``). It means you
 *MUST* run `Burp-UI`_ on the same host that is running your burp server in order
@@ -105,9 +105,19 @@ Alternatively, you can use a `bui-agent <buiagent.html>`__.
 Burp2
 -----
 
-The `burp2 backend <usage.html#burp2>`__ supports only burp 2.0.18 and above.
+The `burp2 backend <advanced_usage.html#burp2>`__ supports only burp 2.0.18 and
+above.
 Some versions are known to contain critical issues resulting in a non-functional
 `Burp-UI`_: 2.0.24, 2.0.26 and 2.0.30
 If you are using an older version of burp2 `Burp-UI`_ will fail to start.
 
+
+Getting started
+---------------
+
+The first thing to do before digging into `Burp-UI`_ is probably to read its
+`architecture <architecture.html>`_ in order to understand how it works.
+Once it's done, you can refer to the `installation <installation.html>`_ page.
+
+
 .. _Burp-UI: https://git.ziirish.me/ziirish/burp-ui

docs/running.rst

@@ -0,0 +1,62 @@
+Running
+=======
+
+There are several ways to run `Burp-UI`_. You can either use the embedded
+Flask's `development server <http://flask.pocoo.org/docs/0.12/server/>`_ or you
+can use any of the `deployement options <http://flask.pocoo.org/docs/0.12/deploying/>`_
+provided by Flask.
+
+.. note:: I personaly focus on ``gunicorn`` support for production deployments
+
+
+Sandboxing
+----------
+
+If you want to play with `Burp-UI`_ to *PoC* it or if you are going to be the
+only user, you can absolutely use the embedded server.
+If you plan to run `Burp-UI`_ in production, then you should go with
+`Gunicorn`_.
+
+Option 1
+^^^^^^^^
+
+You can run the embedded server with the following command:
+
+::
+
+    burp-ui
+
+
+By default, the server listens on *localhost:5000*. You can easily change this
+by adding the ``-- -h x.x.x.x -p yyyy`` options. See `here <installation.html#developer-options>`_
+for details.
+
+Option 2
+^^^^^^^^
+
+Prior to v0.4.0, you could specify the *bind* and *port* option within the
+`Burp-UI`_ configuration file.
+You can still use this behavior by running:
+
+::
+
+    python -m burpui -m legacy [--help]
+
+
+Production
+----------
+
+Like I said earlier, I recommend using `Gunicorn`_ for production deployments.
+You can refer to the dedicated `gunicorn <gunicorn.html>`_ page of this
+documentation to know everything on how to `Burp-UI`_ through `Gunicorn`_.
+
+
+Going further
+-------------
+
+Please refer to the `advanced usage <advanced_usage.html>` page for details on
+how to use/customize `Burp-UI`_.
+
+
+.. _Burp-UI: https://git.ziirish.me/ziirish/burp-ui
+.. _Gunicorn: http://gunicorn.org/

docs/step-by-step.rst

@@ -131,6 +131,14 @@ and init scripts. We copy them at the right place:
     cp /usr/local/share/burpui/etc/burpui.sample.cfg /etc/burp/burpui.cfg
 
 
+We have to edit the default configuration in order to work with a `Burp`_-1
+server:
+
+::
+
+    sed -i "s/^version = .*/version = 1/" /etc/burp/burpui.cfg
+
+
 The default configuration is plug and play for this case, we just have to start
 `Burp-UI`_:
 
@@ -151,7 +159,7 @@ documentation.
 Burp2 server
 ------------
 
-In this scenario, we are going to install a `Burp`_ server version 2.0.28.
+In this scenario, we are going to install a `Burp`_ server version 2.0.54.
 We assume you are using the user *root* to run the following commands.
 
 We begin with the installation of `Burp`_ itself.
@@ -163,18 +171,22 @@ install `Burp-UI`_:
 
     apt-get update
     apt-get install uthash-dev g++ make libssl-dev librsync-dev python2.7-dev \
-    git python-pip libffi-dev libyajl-dev libz-dev
+        git python-pip libffi-dev libyajl-dev libz-dev
 
 
 Now we retrieve the `Burp`_ sources and then we compile and install it:
 
+.. note:: There are now plenty of official and unofficial burp packages. See
+          `here <https://github.com/grke/burp/wiki/Binary-packages>`_ for
+          details.
+
 ::
 
     cd /usr/src
     git clone https://github.com/grke/burp.git
     cd burp
-    git checkout tags/2.0.28
+    git checkout tags/2.0.54
-    ./configure
+    ./configure --prefix=/usr --sysconfdir=/etc/burp --localstatedir=/var
     make
     make install
     # we also install init scripts
@@ -263,14 +275,6 @@ and init scripts. We copy them at the right place:
     cp /usr/local/share/burpui/etc/burpui.sample.cfg /etc/burp/burpui.cfg
 
 
-We have to edit the default configuration in order to work with a `Burp`_-2
-server:
-
-::
-
-    sed -i "s/^version = .*/version = 2/" /etc/burp/burpui.cfg
-
-
 That's it, the other default parameter should be able to handle such a setup.
 We can start `Burp-UI`_:
 
@@ -288,12 +292,6 @@ For further customization, you can refer to the `usage`_ page of this
 documentation.
 
 
-Multiple servers
-----------------
-
-[TODO]
-
-
 .. _Burp-UI: https://git.ziirish.me/ziirish/burp-ui
 .. _Burp: http://burp.grke.net/
-.. _usage: usage.html
+.. _usage: advanced_usage.html

docs/upgrading.rst

@@ -7,6 +7,20 @@ Each section presents major/breaking changes, new requirements and new options.
 For a complete list of changes, you may refer to the
 `CHANGELOG <changelog.html>`_ page.
 
+v0.5.0
+------
+
+- **Breaking** - The *standalone* option has been renamed to *single* to avoid
+  confusion.
+- **Breaking** - The database schema evolved between v0.4.0 and v0.5.0. In order
+  to apply these modifications, you **MUST** run the ``bui-manage db upgrade``
+  command before restarting your `Burp-UI`_ application (if you are using
+  celery, you must restart it too).
+
+  More details on the `Manage <manage.html>`__ and `Celery <celery.html>`__
+  pages.
+
+
 v0.4.0
 ------