aboutsummaryrefslogtreecommitdiffstats
path: root/nbs/export.ipynb
diff options
context:
space:
mode:
authorEugene Triguba <hello@eugenetriguba.com>2026-05-21 01:16:47 +0000
committerEugene Triguba <hello@eugenetriguba.com>2026-05-21 01:28:33 +0000
commitb66a066f74224263f8fab79cb2d8df38baccfa31 (patch)
treeb89d44a12c404dacf638c1d59f12ca2c073aefd5 /nbs/export.ipynb
parentAdd 'export_nb' to pkg __init__ (diff)
downloadnb-b66a066f74224263f8fab79cb2d8df38baccfa31.zip
Remove autoexport of imports and remove noexport directive
Diffstat (limited to 'nbs/export.ipynb')
-rw-r--r--nbs/export.ipynb151
1 files changed, 27 insertions, 124 deletions
diff --git a/nbs/export.ipynb b/nbs/export.ipynb
index d662f7d..f1618f2 100644
--- a/nbs/export.ipynb
+++ b/nbs/export.ipynb
@@ -17,16 +17,25 @@
"metadata": {},
"outputs": [],
"source": [
- "import ast\n",
- "import enum\n",
- "import pathlib\n",
- "import re\n",
+ "#| export\n",
+ "import enum,pathlib,re\n",
"from typing import IO\n",
"\n",
"import nbformat"
]
},
{
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "eb27bbd6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os,tempfile\n",
+ "from io import StringIO"
+ ]
+ },
+ {
"cell_type": "markdown",
"id": "a682b39f",
"metadata": {},
@@ -42,15 +51,12 @@
"To get our notebook-driven source code bootstrapped, we'll first need to be able to convert `ipynb` files to `py` files. Without that, we'd be unable to later import code we want to use in one notebook from another notebook because it's not in a plain importable Python file yet.\n",
"\n",
"To accomplish that, for any given code well, we'll need to know:\n",
- "- Are we exporting it to a Python file?\n",
- "- What Python file are we exporting it to?\n",
+ "- What Python file are we exporting our notebook to?\n",
+ "- Which cells are we exporting to our Python file?\n",
"\n",
"The approach tools like [nbdev](https://nbdev.fast.ai/) or [nbexport](https://github.com/danhorner/nbexport) take is to have certain directives that determine the code export behavior:\n",
"- `#|default_exp: mymodule`: Set the output module for this notebook\n",
- "- `#|export`: Export this cell to the corresponding Python file\n",
- "- `#|noexport`: Prevent export of this cell. Useful to prevent export of something that would be automatically exported.\n",
- "\n",
- "Furthermore, beyond the explicit directives, nbdev and nbexport both use automatic exports of cells with imports in them for less annotation overhead since nearly every import in a notebook is needed by the exported code. This is the primary rationale for a `#|noexport` since sometimes a import is purely for exploratory programming, and you won't use it in the final artifact."
+ "- `#|export`: Export this cell to the corresponding Python file"
]
},
{
@@ -63,8 +69,8 @@
"2. Retrieve all code cells from the notebook\n",
"3. Find all cells that should be exported:\n",
" - Each notebook must have 1 (and only 1) `#| default_exp`. When a `#| default_exp` is found, it must be a single word e.g. `mymodule` or a dotted path e.g. `mymodules.other_module`. If it is a dotted path, the submodules created should have `__init__.py` files also created alongside them.\n",
- " - Any cells with an `#| export` explicit directive or cells that have an import and *don't* have an explicit `#| noexport` directive should be marked for export.\n",
- "4. Write out each cell sequentially into the Python file, stripping out any directives. When each cell is written out, it should have a comment above it describing the cell ID it came from and the path to the notebook it came from."
+ " - Any cells with an `#| export` explicit directive should be marked for export.\n",
+ "4. Write out each cell sequentially into the Python file, stripping out any directives. When each cell is written out, it should have a comment above it describing the cell ID it came from."
]
},
{
@@ -80,15 +86,7 @@
"id": "037fb28c",
"metadata": {},
"source": [
- "To read in `ipynb` files, we can use a library provided Jupyter called `nbformat`:"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "39f9ece6",
- "metadata": {},
- "source": [
- "And simply because we'll be reading in this notebook as our working copy while developing this export behavior, let's add a raw cell just so we have an example of all 3 cell types:"
+ "To read in `ipynb` files, we can use a library provided Jupyter called `nbformat`. And simply because we'll be reading in this notebook as our working copy while developing this export behavior, let's add a raw cell just so we have an example of all 3 cell types."
]
},
{
@@ -494,25 +492,6 @@
},
{
"cell_type": "markdown",
- "id": "50c2d8ff",
- "metadata": {},
- "source": [
- "#### Supporting the noexport directive"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "54459016",
- "metadata": {},
- "outputs": [],
- "source": [
- "#| export\n",
- "DIRECTIVE_NOEXPORT_RE = _mk_directive_re('noexport')"
- ]
- },
- {
- "cell_type": "markdown",
"id": "a7a778d2",
"metadata": {},
"source": [
@@ -527,51 +506,7 @@
"outputs": [],
"source": [
"#| export\n",
- "DIRECTIVES = [DIRECTIVE_DEFAULT_EXP_RE, DIRECTIVE_EXPORT_RE, DIRECTIVE_NOEXPORT_RE]"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "020f82b0",
- "metadata": {},
- "source": [
- "### Matching Python Imports"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7863f491",
- "metadata": {},
- "source": [
- "Since we know we're working with Python code cells, we can use the `ast` module to robustly detect if any `import`s are in the code cell."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "10232dd8",
- "metadata": {},
- "outputs": [],
- "source": [
- "#| export\n",
- "def has_import(src:str)->bool:\n",
- " \"Check whether a block of Python source code has any import statements.\"\n",
- " try: return any(isinstance(node, (ast.Import, ast.ImportFrom)) for node in ast.walk(ast.parse(src)))\n",
- " except SyntaxError: return False\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "a61b2562",
- "metadata": {},
- "outputs": [],
- "source": [
- "assert has_import('1 + 1\\nimport re\\na = 5\\n')\n",
- "assert has_import('from re import Pattern\\na = 5\\n')\n",
- "assert not has_import('# import re\\n')\n",
- "assert not has_import('a = 5\\n')\n",
- "assert not has_import('max(1, 10)\\n')"
+ "DIRECTIVES = [DIRECTIVE_DEFAULT_EXP_RE, DIRECTIVE_EXPORT_RE]"
]
},
{
@@ -591,8 +526,7 @@
"\n",
"**Algorithm:**\n",
"1. Always check if there is a `default_exp` directive. There must be one (and only one) `default_exp` directive.\n",
- "2. Check if an import is present. If present, check if a noexport directive is present.\n",
- "3. If no import is present, check for an explicit export directive."
+ "3. Check for an explicit export directive."
]
},
{
@@ -617,7 +551,6 @@
"source": [
"assert has_directive('#| default_exp: bootstrap')\n",
"assert has_directive('#| export')\n",
- "assert has_directive('#| noexport')\n",
"assert has_directive('max(1,20)\\n#| export')\n",
"assert not has_directive('# comment\\nmax(1, 20)\\n')"
]
@@ -644,7 +577,6 @@
"outputs": [],
"source": [
"assert strip_directives('#| export\\na=5') == 'a=5'\n",
- "assert strip_directives('#| noexport\\na=5') == 'a=5'\n",
"assert strip_directives('#| default_exp: bootstrap\\na=5') == 'a=5'\n",
"assert strip_directives(['a=10', 'a+=10', '#|export']) == 'a=10\\na+=10'"
]
@@ -659,11 +591,7 @@
"#| export\n",
"def is_cell_exported(cell:dict)->bool:\n",
" \"Check whether the code cell should be exported.\"\n",
- " src = cell['source']\n",
- " lines = src.splitlines()\n",
- " if any(DIRECTIVE_NOEXPORT_RE.match(l) for l in lines): return False\n",
- " if any(DIRECTIVE_EXPORT_RE.match(l) for l in lines): return True\n",
- " return has_import(src)"
+ " return any(DIRECTIVE_EXPORT_RE.match(l) for l in cell['source'].splitlines())"
]
},
{
@@ -674,12 +602,10 @@
"outputs": [],
"source": [
"mk_cell = lambda s,id=1: dict(id=id,source=s,cell_type=CellType.CODE)\n",
- "assert is_cell_exported(mk_cell('import re'))\n",
- "assert is_cell_exported(mk_cell('from re import Pattern'))\n",
"assert is_cell_exported(mk_cell('#| export'))\n",
- "assert not is_cell_exported(mk_cell('#| noexport'))\n",
- "assert not is_cell_exported(mk_cell('#| noexport\\n#| export'))\n",
- "assert not is_cell_exported(mk_cell('#| export\\n#| noexport'))"
+ "assert is_cell_exported(mk_cell('import re\\n#| export'))\n",
+ "assert not is_cell_exported(mk_cell('import re'))\n",
+ "assert not is_cell_exported(mk_cell('1+1\\nfrom re import Pattern'))"
]
},
{
@@ -801,18 +727,6 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "c044d6d4",
- "metadata": {},
- "outputs": [],
- "source": [
- "#| noexport\n",
- "import os\n",
- "import tempfile"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
"id": "e19b3289",
"metadata": {},
"outputs": [],
@@ -874,17 +788,6 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "c85bc348",
- "metadata": {},
- "outputs": [],
- "source": [
- "#| noexport\n",
- "from io import StringIO"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
"id": "55f57bee",
"metadata": {},
"outputs": [],
@@ -906,9 +809,9 @@
"source": [
"cells = [\n",
" nbformat.v4.new_code_cell('#| default_exp: testmod'),\n",
- " nbformat.v4.new_code_cell('import os'),\n",
+ " nbformat.v4.new_code_cell('#| export \\nimport os'),\n",
" nbformat.v4.new_code_cell('#| export\\ndef foo(): pass'),\n",
- " nbformat.v4.new_code_cell('#| noexport\\nimport pathlib'),\n",
+ " nbformat.v4.new_code_cell('import pathlib'),\n",
"]\n",
"nb = mk_nb(cells)\n",
"src = f\"\"\"# AUTOGENERATED! DO NOT EDIT!\n",