diff options
| author | Eugene Triguba <hello@eugenetriguba.com> | 2026-05-21 01:16:47 +0000 |
|---|---|---|
| committer | Eugene Triguba <hello@eugenetriguba.com> | 2026-05-21 01:28:33 +0000 |
| commit | b66a066f74224263f8fab79cb2d8df38baccfa31 (patch) | |
| tree | b89d44a12c404dacf638c1d59f12ca2c073aefd5 /nbs/export.ipynb | |
| parent | Add 'export_nb' to pkg __init__ (diff) | |
| download | nb-b66a066f74224263f8fab79cb2d8df38baccfa31.zip | |
Remove autoexport of imports and remove noexport directive
Diffstat (limited to 'nbs/export.ipynb')
| -rw-r--r-- | nbs/export.ipynb | 151 |
1 files changed, 27 insertions, 124 deletions
diff --git a/nbs/export.ipynb b/nbs/export.ipynb index d662f7d..f1618f2 100644 --- a/nbs/export.ipynb +++ b/nbs/export.ipynb @@ -17,16 +17,25 @@ "metadata": {}, "outputs": [], "source": [ - "import ast\n", - "import enum\n", - "import pathlib\n", - "import re\n", + "#| export\n", + "import enum,pathlib,re\n", "from typing import IO\n", "\n", "import nbformat" ] }, { + "cell_type": "code", + "execution_count": null, + "id": "eb27bbd6", + "metadata": {}, + "outputs": [], + "source": [ + "import os,tempfile\n", + "from io import StringIO" + ] + }, + { "cell_type": "markdown", "id": "a682b39f", "metadata": {}, @@ -42,15 +51,12 @@ "To get our notebook-driven source code bootstrapped, we'll first need to be able to convert `ipynb` files to `py` files. Without that, we'd be unable to later import code we want to use in one notebook from another notebook because it's not in a plain importable Python file yet.\n", "\n", "To accomplish that, for any given code well, we'll need to know:\n", - "- Are we exporting it to a Python file?\n", - "- What Python file are we exporting it to?\n", + "- What Python file are we exporting our notebook to?\n", + "- Which cells are we exporting to our Python file?\n", "\n", "The approach tools like [nbdev](https://nbdev.fast.ai/) or [nbexport](https://github.com/danhorner/nbexport) take is to have certain directives that determine the code export behavior:\n", "- `#|default_exp: mymodule`: Set the output module for this notebook\n", - "- `#|export`: Export this cell to the corresponding Python file\n", - "- `#|noexport`: Prevent export of this cell. Useful to prevent export of something that would be automatically exported.\n", - "\n", - "Furthermore, beyond the explicit directives, nbdev and nbexport both use automatic exports of cells with imports in them for less annotation overhead since nearly every import in a notebook is needed by the exported code. This is the primary rationale for a `#|noexport` since sometimes a import is purely for exploratory programming, and you won't use it in the final artifact." + "- `#|export`: Export this cell to the corresponding Python file" ] }, { @@ -63,8 +69,8 @@ "2. Retrieve all code cells from the notebook\n", "3. Find all cells that should be exported:\n", " - Each notebook must have 1 (and only 1) `#| default_exp`. When a `#| default_exp` is found, it must be a single word e.g. `mymodule` or a dotted path e.g. `mymodules.other_module`. If it is a dotted path, the submodules created should have `__init__.py` files also created alongside them.\n", - " - Any cells with an `#| export` explicit directive or cells that have an import and *don't* have an explicit `#| noexport` directive should be marked for export.\n", - "4. Write out each cell sequentially into the Python file, stripping out any directives. When each cell is written out, it should have a comment above it describing the cell ID it came from and the path to the notebook it came from." + " - Any cells with an `#| export` explicit directive should be marked for export.\n", + "4. Write out each cell sequentially into the Python file, stripping out any directives. When each cell is written out, it should have a comment above it describing the cell ID it came from." ] }, { @@ -80,15 +86,7 @@ "id": "037fb28c", "metadata": {}, "source": [ - "To read in `ipynb` files, we can use a library provided Jupyter called `nbformat`:" - ] - }, - { - "cell_type": "markdown", - "id": "39f9ece6", - "metadata": {}, - "source": [ - "And simply because we'll be reading in this notebook as our working copy while developing this export behavior, let's add a raw cell just so we have an example of all 3 cell types:" + "To read in `ipynb` files, we can use a library provided Jupyter called `nbformat`. And simply because we'll be reading in this notebook as our working copy while developing this export behavior, let's add a raw cell just so we have an example of all 3 cell types." ] }, { @@ -494,25 +492,6 @@ }, { "cell_type": "markdown", - "id": "50c2d8ff", - "metadata": {}, - "source": [ - "#### Supporting the noexport directive" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "54459016", - "metadata": {}, - "outputs": [], - "source": [ - "#| export\n", - "DIRECTIVE_NOEXPORT_RE = _mk_directive_re('noexport')" - ] - }, - { - "cell_type": "markdown", "id": "a7a778d2", "metadata": {}, "source": [ @@ -527,51 +506,7 @@ "outputs": [], "source": [ "#| export\n", - "DIRECTIVES = [DIRECTIVE_DEFAULT_EXP_RE, DIRECTIVE_EXPORT_RE, DIRECTIVE_NOEXPORT_RE]" - ] - }, - { - "cell_type": "markdown", - "id": "020f82b0", - "metadata": {}, - "source": [ - "### Matching Python Imports" - ] - }, - { - "cell_type": "markdown", - "id": "7863f491", - "metadata": {}, - "source": [ - "Since we know we're working with Python code cells, we can use the `ast` module to robustly detect if any `import`s are in the code cell." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10232dd8", - "metadata": {}, - "outputs": [], - "source": [ - "#| export\n", - "def has_import(src:str)->bool:\n", - " \"Check whether a block of Python source code has any import statements.\"\n", - " try: return any(isinstance(node, (ast.Import, ast.ImportFrom)) for node in ast.walk(ast.parse(src)))\n", - " except SyntaxError: return False\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a61b2562", - "metadata": {}, - "outputs": [], - "source": [ - "assert has_import('1 + 1\\nimport re\\na = 5\\n')\n", - "assert has_import('from re import Pattern\\na = 5\\n')\n", - "assert not has_import('# import re\\n')\n", - "assert not has_import('a = 5\\n')\n", - "assert not has_import('max(1, 10)\\n')" + "DIRECTIVES = [DIRECTIVE_DEFAULT_EXP_RE, DIRECTIVE_EXPORT_RE]" ] }, { @@ -591,8 +526,7 @@ "\n", "**Algorithm:**\n", "1. Always check if there is a `default_exp` directive. There must be one (and only one) `default_exp` directive.\n", - "2. Check if an import is present. If present, check if a noexport directive is present.\n", - "3. If no import is present, check for an explicit export directive." + "3. Check for an explicit export directive." ] }, { @@ -617,7 +551,6 @@ "source": [ "assert has_directive('#| default_exp: bootstrap')\n", "assert has_directive('#| export')\n", - "assert has_directive('#| noexport')\n", "assert has_directive('max(1,20)\\n#| export')\n", "assert not has_directive('# comment\\nmax(1, 20)\\n')" ] @@ -644,7 +577,6 @@ "outputs": [], "source": [ "assert strip_directives('#| export\\na=5') == 'a=5'\n", - "assert strip_directives('#| noexport\\na=5') == 'a=5'\n", "assert strip_directives('#| default_exp: bootstrap\\na=5') == 'a=5'\n", "assert strip_directives(['a=10', 'a+=10', '#|export']) == 'a=10\\na+=10'" ] @@ -659,11 +591,7 @@ "#| export\n", "def is_cell_exported(cell:dict)->bool:\n", " \"Check whether the code cell should be exported.\"\n", - " src = cell['source']\n", - " lines = src.splitlines()\n", - " if any(DIRECTIVE_NOEXPORT_RE.match(l) for l in lines): return False\n", - " if any(DIRECTIVE_EXPORT_RE.match(l) for l in lines): return True\n", - " return has_import(src)" + " return any(DIRECTIVE_EXPORT_RE.match(l) for l in cell['source'].splitlines())" ] }, { @@ -674,12 +602,10 @@ "outputs": [], "source": [ "mk_cell = lambda s,id=1: dict(id=id,source=s,cell_type=CellType.CODE)\n", - "assert is_cell_exported(mk_cell('import re'))\n", - "assert is_cell_exported(mk_cell('from re import Pattern'))\n", "assert is_cell_exported(mk_cell('#| export'))\n", - "assert not is_cell_exported(mk_cell('#| noexport'))\n", - "assert not is_cell_exported(mk_cell('#| noexport\\n#| export'))\n", - "assert not is_cell_exported(mk_cell('#| export\\n#| noexport'))" + "assert is_cell_exported(mk_cell('import re\\n#| export'))\n", + "assert not is_cell_exported(mk_cell('import re'))\n", + "assert not is_cell_exported(mk_cell('1+1\\nfrom re import Pattern'))" ] }, { @@ -801,18 +727,6 @@ { "cell_type": "code", "execution_count": null, - "id": "c044d6d4", - "metadata": {}, - "outputs": [], - "source": [ - "#| noexport\n", - "import os\n", - "import tempfile" - ] - }, - { - "cell_type": "code", - "execution_count": null, "id": "e19b3289", "metadata": {}, "outputs": [], @@ -874,17 +788,6 @@ { "cell_type": "code", "execution_count": null, - "id": "c85bc348", - "metadata": {}, - "outputs": [], - "source": [ - "#| noexport\n", - "from io import StringIO" - ] - }, - { - "cell_type": "code", - "execution_count": null, "id": "55f57bee", "metadata": {}, "outputs": [], @@ -906,9 +809,9 @@ "source": [ "cells = [\n", " nbformat.v4.new_code_cell('#| default_exp: testmod'),\n", - " nbformat.v4.new_code_cell('import os'),\n", + " nbformat.v4.new_code_cell('#| export \\nimport os'),\n", " nbformat.v4.new_code_cell('#| export\\ndef foo(): pass'),\n", - " nbformat.v4.new_code_cell('#| noexport\\nimport pathlib'),\n", + " nbformat.v4.new_code_cell('import pathlib'),\n", "]\n", "nb = mk_nb(cells)\n", "src = f\"\"\"# AUTOGENERATED! DO NOT EDIT!\n", |