Another documentation pass (#63)

* Updated the introduction of the documentation

* Added contributing and release files

* Updated introduction and fixed typos in some notebooks

* Added TF to install_requires if not installed. Adjusted install guide and fixed typos in contributing.md

* Added log1pe transforms to keep ell > 1e-3 for stability to firststeps notebook too

* Updated contributing

* Fixed minor typo

Author: icouckuy

CONTRIBUTING.md

@@ -0,0 +1,38 @@
+# Contributing to GPflowOpt
+This file contains notes for potential contributors to GPflowOpt, as well as some notes that may be helpful for maintenance. As part of the GPflow organisation, GPflowOpt follows the same philosophy and principles with regards to scope and code quality as GPflow.
+
+## Project scope
+We do welcome contributions to GPflowOpt. However, the project is deliberately of limited scope, to try to ensure a high quality codebase: if you'd like to contribute a feature, please raise discussion via a GitHub issue.
+
+Due to limited scope we may not be able to review and merge every feature, however useful it may be. Particularly large contributions or changes to core code are harder to justify against the scope of the project or future development plans. For these contributions, we suggest you publish them as a separate package based on GPflowOpt's interfaces. We can link to your project from an issue discussing the topic or within the repository. Discussing a possible contribution in an issue should give an indication to how broadly it is supported to bring it into the codebase.
+
+## Code Style
+ - Python code should follow roughly the [PEP 8](https://www.python.org/dev/peps/pep-0008/) style. We allow exceptions for, e.g., capital (single-letter) variable names to correspond to the notation of a paper (matrices, vectors, etc.). To help with this, we suggest using a plugin for your editor (we use pycharm).
+ - Practise good code as far as is reasonable. Simpler is usually better. Avoid using complicated language features. Reading the existing GPflowOpt code should give a good idea of the expected style.
+
+## Pull requests and the master branch
+All code that is destined for the master branch of GPflowOpt goes through a PR and will be reviewed. Only a small number of people can merge PRs onto the master branch (currently Joachim van der Herten and Ivo Couckuyt).
+
+## Tests and continuous integration
+GPflowOpt is 99% covered by the testing suite. We expect changes to code to pass these tests, and for new code to be covered by new tests. Currently, tests are run by travis (python 2.7, 3.5 and 3.6), coverage is reported by codecov.
+
+By default, all tests are run on Travis except for the most expensive notebooks.
+
+## Documentation
+GPflowOpt's documentation is not comprehensive, but covers enough to get users started. We expect that new features have documentation that can help other get up to speed. The docs are mostly Jupyter notebooks that compile into html via sphinx, using nbsphinx.
+
+## Keeping up with GPflow and TensorFlow
+
+GPflowOpt currently tries to keep up with the GPflow master, though at some point we will start depending on the latest released version. Hence, GPflowOpt also adheres to the api of the TensorFlow version as required by GPflow. In practice this hopefully means we will support the latest (stable) TensorFlow, which is supported by GPflow. Any change in the supported version of GPflow or TensorFlow will bump the minor version number of GPflowOpt.
+
+Changing the minimum required version of TensorFlow that we're compatible with requires a few tasks:
+ - update versions in `setup.py`
+ - update versions used on travis via `.travis.yml`
+ - update version ussed by readthedocs.org via `docsrequire.txt`
+ - Increment the GPflowOpt version (see below). 
+
+## Version numbering
+The main purpose of versioning GPflowOpt is user convenience: to keep the number of releases down, we try to combine seversal PRs into one increment. As we work towards something that we might call 1.0, including changes to the GPflowOpt API. Minor version bumps (X.1) are used for updates to follow a new GPflow or TensorFlow API, or introduce incremental new features.
+When incrementing the version number, the following tasks are required:
+ - Update the version in `GPflowOpt/_version.py`
+ - Add a note to `RELEASE.md`

RELEASE.md

@@ -0,0 +1 @@
+Pre-release

doc/source/intro.rst

@@ -2,31 +2,39 @@
 Introduction
 ------------
 
-GPflowOpt is a library for Bayesian Optimization with GPflow. It makes use of TensorFlow for computation of acquisition functions,
-to offer scalability, and avoid implementation of gradients. The package was created, and is currently maintained
-by `Joachim van der Herten <http://sumo.intec.ugent.be/jvanderherten>`_ and `Ivo Couckuyt <http://sumo.intec.ugent.be/icouckuy>`_
+`GPflowOpt <https://github.com/GPflow/GPflowOpt/>`_ is a library for Bayesian Optimization with `GPflow <https://github.com/GPflow/GPflow/>`_.
+It makes use of TensorFlow for computation of acquisition functions, to offer scalability, and avoid implementation of gradients.
+The package was created, and is currently maintained by `Joachim van der Herten <http://sumo.intec.ugent.be/jvanderherten>`_ and `Ivo Couckuyt <http://sumo.intec.ugent.be/icouckuy>`_
 
 Currently the software is pre-release and under construction, hence it lacks a lot of functionality and testing. This documentation
 is also incomplete and under development. The project is open source: if you feel you have some relevant skills and are interested in
 contributing then please contact us on `GitHub <https://github.com/GPflow/GPflowOpt>`_ by opening an issue or pull request.
 
 Install
 --------
-1. Install TensorFlow
+1. Install package
 
-You will need version 1.0 or above. We find that for many users pip installation is the fastest way to get going.
-``pip install tensorflow``
+A straightforward way to install GPflowOpt is to clone its repository and run
 
-For alternative installations, please see the instructions on the main `TensorFlow webpage <https://www.tensorflow.org/install/>`_.
+``pip install . --process-dependency-links``
 
-2. Install package
+in the root folder. This also installs required dependencies including TensorFlow.
+For alternative TensorFlow installations (e.g., gpu), please see the instructions on the main `TensorFlow webpage <https://www.tensorflow.org/install/>`_.
 
-GPflowOpt is a pure python library for now, so you could just add it to your path (we use ``python setup.py develop``). A straightforward way to install GPflowOpt including all of its dependencies: ``pip install . --process-dependency-links``
-A different way to install the package: ``python setup.py install`` (untested)
+2. Development
 
-You can run the tests with ``python setup.py test``.
+GPflowOpt is a pure python library so you could just add it to your python path. We use
 
-To build the documentation, first install extra dependencies with ``pip install .[docs]``, then proceed with ``python setup.py build_sphinx``.
+``pip  install -e . --process-dependency-links``
+
+3. Testing and documentation
+
+The tests require some additional dependencies that need to be installed first with
+``pip install -e .[test]``. Afterwards the tests can be run with ``python setup.py test``.
+
+Similarly, to build the documentation,
+first install the extra dependencies with ``pip install -e .[docs]``.
+Then proceed with ``python setup.py build_sphinx``.
 
 Getting started
 ---------------
@@ -39,4 +47,4 @@ For more advanced use cases have a look at the other :ref:`tutorial <tutorials>`
 Acknowledgements
 -----------------
 Joachim van der Herten and Ivo Couckuyt are Ghent University - imec postdoctoral fellows. Ivo Couckuyt is supported
-by FWO Vlaanderen.
+by FWO Vlaanderen.

doc/source/notebooks/firststeps.ipynb

@@ -16,11 +16,9 @@
    "source": [
     "## Introduction\n",
     "\n",
-    "Bayesian optimization is particularly useful for expensive optimization problems. This includes optimization problems where the objective (and constraints) are time-consuming to evaluate: measurements, engineering simulations, cross-hyperparameter optimization of deep learning models, etc. Another area where Bayesian optimization may provide a benefit is in the presence of (a lot of) noise.\n",
+    "Bayesian optimization is particularly useful for expensive optimization problems. This includes optimization problems where the objective (and constraints) are time-consuming to evaluate: measurements, engineering simulations, hyperparameter optimization of deep learning models, etc. Another area where Bayesian optimization may provide a benefit is in the presence of (a lot of) noise. If your problem does not satisfy these requirements other optimization algorithms might be better suited.\n",
     "\n",
-    "If your problem does not satisfy these requirements other optimization algorithms might be better suited.\n",
-    "\n",
-    "To setup a basic Bayesian optimization you have to:\n",
+    "To setup a Bayesian optimization scheme with GPflowOpt you have to:\n",
     "\n",
     "- define your objective and specify the optimization domain\n",
     "- setup a GPflow model and choose an acquisition function\n",
@@ -45,7 +43,7 @@
        "<table id='domain' width=100%><tr><td>Name</td><td>Type</td><td>Values</td></tr><tr><td>x1</td><td>Continuous</td><td>[-2.  2.]</td></tr><tr><td>x2</td><td>Continuous</td><td>[-1.  2.]</td></tr></table>"
       ],
       "text/plain": [
-       "<GPflowOpt.domain.Domain at 0x21ea69589b0>"
+       "<GPflowOpt.domain.Domain at 0x7f1f613eef60>"
       ]
      },
      "execution_count": 1,
@@ -60,7 +58,6 @@
     "\n",
     "def fx(X):\n",
     "    X = np.atleast_2d(X)\n",
-    "    # Return objective & gradient\n",
     "    return np.sum(np.square(X), axis=1)[:, None]\n",
     "\n",
     "domain = ContinuousParameter('x1', -2, 2) + ContinuousParameter('x2', -1, 2)\n",
@@ -83,12 +80,15 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Warning: optimization restart 5/5 failed\n",
-      "     fun: array([  1.69219493e-05])\n",
+      "Warning: optimization restart 1/5 failed\n",
+      "Warning: optimization restart 2/5 failed\n",
+      "Warning: optimization restart 3/5 failed\n",
+      "Warning: optimization restart 2/5 failed\n",
+      "     fun: array([ 0.01])\n",
       " message: 'OK'\n",
       "    nfev: 15\n",
       " success: True\n",
-      "       x: array([[ -9.60370839e-05,   4.11250850e-03]])\n"
+      "       x: array([[ 0. , -0.1]])\n"
      ]
     }
    ],
@@ -104,6 +104,7 @@
     "X = lhd.generate()\n",
     "Y = fx(X)\n",
     "model = GPflow.gpr.GPR(X, Y, GPflow.kernels.Matern52(2, ARD=True))\n",
+    "model.kern.lengthscales.transform = GPflow.transforms.Log1pe(1e-3)\n",
     "\n",
     "# Now create the Bayesian Optimizer\n",
     "alpha = ExpectedImprovement(model)\n",
@@ -119,7 +120,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "That's all! Your objective function has now been optimized over 15 iterations."
+    "That's all! Your objective function has now been optimized for 15 iterations."
    ]
   }
  ],