{ "cells": [ { "cell_type": "code", "execution_count": null, "id": "d42a9120-8d70-4b10-8d6c-b59692fd03ad", "metadata": {}, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "code", "execution_count": null, "id": "11cb390f-b0f4-438d-b524-db7242d22ee9", "metadata": {}, "outputs": [], "source": [ "#| default_exp: test" ] }, { "cell_type": "code", "execution_count": null, "id": "dd03783f-252c-40f5-b92f-0f3f9278ce09", "metadata": {}, "outputs": [], "source": [ "#| export\n", "import ast,copy,contextlib,inspect,textwrap,traceback\n", "from typing import Any,Generator\n", "from dataclasses import dataclass" ] }, { "cell_type": "code", "execution_count": null, "id": "2cf5d527", "metadata": {}, "outputs": [], "source": [ "from nb.core import pubattr" ] }, { "cell_type": "markdown", "id": "5b8abf6a-c94a-43e5-a1d1-81b1a1d64656", "metadata": {}, "source": [ "# Testing\n", "> Utilities for testing within notebooks" ] }, { "cell_type": "markdown", "id": "12e34b1f", "metadata": {}, "source": [ "## Re-constructing source code and program state during runtime" ] }, { "cell_type": "markdown", "id": "93a3d5fe", "metadata": {}, "source": [ "When writing test libraries, it can be incredibly useful to understand what the call site source code looks like.\n", "\n", "For instance, let's say you call `equal(x in [1, 2, 3], should_exist)`. When we're in `equal`, the arguments have already been evaluated and we simply have two booleans as arguments, each `True` or `False`. If this test fails, it likely isn't *that* helpful to see `False != True`. \n", "\n", "Instead, it is likely much *more* helpful to see something like `x in [1, 2, 3] != should_exist` where `x = 4` and `should_exist=True`." ] }, { "cell_type": "code", "execution_count": null, "id": "a6c67919", "metadata": {}, "outputs": [], "source": [ "#| export\n", "def get_call_args(frames_up: int = 1) -> tuple[list[str], dict[str,Any]]:\n", " \"Get the source code expressions of the caller's arguments.\"\n", " fm_info = inspect.getouterframes(inspect.currentframe())[frames_up+1]\n", " src, _ = inspect.getsourcelines(fm_info.frame)\n", " srcl = src[fm_info.positions.lineno - 1:fm_info.positions.end_lineno]\n", " if len(srcl) == 0:\n", " raise RuntimeError(\n", " \"`get_call_args` is designed to be called from within a function, \" +\n", " \"not at the top-level. If you want to call it at the top level, pass `frames_up=0`.\"\n", " )\n", " elif len(srcl) == 1:\n", " fn_src = srcl[0][fm_info.positions.col_offset:fm_info.positions.end_col_offset]\n", " else:\n", " fn_src = \"\".join(\n", " [srcl[0][fm_info.positions.col_offset:]] +\n", " srcl[1:-1] +\n", " [srcl[-1][:fm_info.positions.end_col_offset]]\n", " )\n", " call_node = ast.parse(fn_src).body[0].value\n", " args_src = [ast.unparse(arg) for arg in call_node.args]\n", " kwargs_src = {kw.arg: ast.unparse(kw.value) for kw in call_node.keywords}\n", " return args_src, kwargs_src" ] }, { "cell_type": "markdown", "id": "487c29d9", "metadata": {}, "source": [ "Because `get_call_args()` is expected to be used from within a function, it generally will not work properly by default if you call it from the top-level." ] }, { "cell_type": "code", "execution_count": null, "id": "bbe2859c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "`get_call_args` is designed to be called from within a function, not at the top-level. If you want to call it at the top level, pass `frames_up=0`.\n" ] } ], "source": [ "try: get_call_args(); assert False, \"should have raised\"\n", "except RuntimeError as e: print(e)" ] }, { "cell_type": "markdown", "id": "ce788841", "metadata": {}, "source": [ "However, you _can_ have it work if you pass `frames_up=0` (i.e. the current frame we're calling `get_call_args()` from rather than the caller frame)." ] }, { "cell_type": "code", "execution_count": null, "id": "1f3b1abb", "metadata": {}, "outputs": [], "source": [ "args, kwargs = get_call_args(frames_up=0)\n", "assert args == [], f\"Got {args}\"\n", "assert kwargs == {\"frames_up\": \"0\"}, f\"Got {kwargs}\"" ] }, { "cell_type": "markdown", "id": "5a48fa09", "metadata": {}, "source": [ "When you're in a function, it works as expected." ] }, { "cell_type": "code", "execution_count": null, "id": "b7730389", "metadata": {}, "outputs": [], "source": [ "def f(a, b=1):\n", " return get_call_args()" ] }, { "cell_type": "code", "execution_count": null, "id": "19890aa0", "metadata": {}, "outputs": [], "source": [ "args, kwargs = f(1+1,b=max(10, 20))\n", "assert args == [\"1 + 1\"], f\"Got {args}\"\n", "assert kwargs == {\"b\": \"max(10, 20)\"}, f\"Got {kwargs}\"" ] }, { "cell_type": "markdown", "id": "b9aba848", "metadata": {}, "source": [ "It also works with multi-line function calls.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "b8b7195c", "metadata": {}, "outputs": [], "source": [ "args, kwargs = f(\n", " 1,\n", " b=10,\n", " \n", " \n", ")\n", "assert args == [\"1\"], f\"Got {args}\"\n", "assert kwargs == {\"b\": \"10\"}, f\"Got {kwargs}\"" ] }, { "cell_type": "markdown", "id": "f93a8882", "metadata": {}, "source": [ "If you use it with a function within a function, you'll get the arguments for the inner function.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "ce0666f5", "metadata": {}, "outputs": [], "source": [ "def nested_f(a, b=1):\n", " def inner_f(c, d, e=1):\n", " return get_call_args()\n", " return inner_f(5 * 5, 10 / 10)" ] }, { "cell_type": "code", "execution_count": null, "id": "3555f6cf", "metadata": {}, "outputs": [], "source": [ "args, kwargs = nested_f(1)\n", "assert args == [\"5 * 5\", \"10 / 10\"], f\"Got {args}\"\n", "assert kwargs == {}, f\"Got {kwargs}\"" ] }, { "cell_type": "markdown", "id": "97d47759", "metadata": {}, "source": [ "However, you can still get the outer function arguments with the `frames_up` keyword argument." ] }, { "cell_type": "code", "execution_count": null, "id": "2a42fc2b", "metadata": {}, "outputs": [], "source": [ "def nested_f(a, b=1):\n", " def inner_f(c, d, e=1):\n", " return get_call_args(frames_up=2)\n", " return inner_f(5 * 5, 10 / 10)" ] }, { "cell_type": "code", "execution_count": null, "id": "b3687819", "metadata": {}, "outputs": [], "source": [ "args, kwargs = nested_f(1)\n", "assert args == [\"1\"], f\"Got {args}\"\n", "assert kwargs == {}, f\"Got {kwargs}\"" ] }, { "cell_type": "markdown", "id": "5d991fd4", "metadata": {}, "source": [ "## Assertions" ] }, { "cell_type": "code", "execution_count": null, "id": "623a480c", "metadata": {}, "outputs": [], "source": [ "#| export\n", "@dataclass\n", "class ExceptionInfo:\n", " type_: type[Exception] | None = None\n", " value: Exception | None = None\n", " traceback: str | None = None" ] }, { "cell_type": "code", "execution_count": null, "id": "0ad5fcef-f04d-469d-8ad1-94d3a98694e4", "metadata": {}, "outputs": [], "source": [ "#| export\n", "@contextlib.contextmanager\n", "def raises(\n", " exc: type[Exception] = Exception, # The exception type to check is raised.\n", " contains: str | None = None, # Optionally, check whether the raised exception contains this message.\n", ") -> Generator[ExceptionInfo, None, None]:\n", " \"Asserts that the body of the context manager raises `exc`.\"\n", " raised = True\n", " try:\n", " info = ExceptionInfo()\n", " yield info\n", " raised = False\n", " except exc as e:\n", " info.type_ = type(e)\n", " info.value = e\n", " info.traceback = traceback.format_tb(e.__traceback__)\n", " if contains and contains not in str(e):\n", " raise AssertionError(f\"Raised {e} but message does not contain {contains}\") from e\n", " if not raised:\n", " raise AssertionError(f\"Did not raise {e}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "7c478098-7a09-403c-a62b-3c34cf0f7bab", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ExceptionInfo(type_=, value=Exception('abc'), traceback=[' File \"/tmp/ipykernel_572/3259713399.py\", line 10, in raises\\n yield info\\n', ' File \"/tmp/ipykernel_572/3922684010.py\", line 2, in \\n raise Exception(\"abc\")\\n'])\n" ] } ], "source": [ "with raises() as e:\n", " raise Exception(\"abc\")\n", "print(e)" ] }, { "cell_type": "code", "execution_count": null, "id": "2b503871-3f39-4013-a7ae-f5143af3138e", "metadata": {}, "outputs": [], "source": [ "#| export\n", "def equal(a, b):\n", " \"Assert that a == b.\"\n", " args_src, _ = get_call_args()\n", " if a != b:\n", " msg = (\n", " f\"FAIL: {args_src[0]} != {args_src[1]}\\n\" +\n", " ' ' * (len(AssertionError.__name__)+2) +\n", " f\" {a!r} != {b!r}\"\n", " )\n", " raise AssertionError(msg)" ] }, { "cell_type": "code", "execution_count": null, "id": "cc7872bb", "metadata": {}, "outputs": [], "source": [ "with raises(exc=AssertionError) as e:\n", " equal(1, 2)" ] }, { "cell_type": "code", "execution_count": null, "id": "7a1393a7-6a89-4c5d-a8a7-51e0fd46d190", "metadata": {}, "outputs": [], "source": [ "import nb; nb.export_nb('test.ipynb', src_dir='../nb/')" ] } ], "metadata": {}, "nbformat": 4, "nbformat_minor": 5 }