ToolBoxV2: The App
Class¶
The App
class is the central singleton instance in ToolBoxV2, responsible for managing the application's lifecycle, configuration, module loading, and core functionalities. It's typically accessed via the get_app()
utility function.
Initialization¶
The App
instance is initialized with a prefix
and AppArgs
(command-line arguments).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
Key Initialization Steps:¶
-
System & Paths:
- Determines the operating system (
system_flag
). - Sets the
start_dir
to the application's root directory. - Resolves the
prefix
:- If no prefix is provided, it attempts to load the last used prefix from
.data/last-app-prefix.txt
. - If a prefix is provided, it's saved to this file for future use.
- If no prefix is provided, it attempts to load the last used prefix from
- Constructs the
app_id
(e.g.,prefix-hostname
). - Sets up
data_dir
,config_dir
, andinfo_dir
based on theapp_id
(e.g.,./.data/prefix-hostname/
). - Sets up
appdata
directory (OS-specific application data folder).
- Determines the operating system (
-
Logging:
- Initializes a logger (
app.logger
). The logging level and output (terminal/file) can vary based on theprefix
(e.g., "test", "live", "debug") and the--debug
CLI argument.
- Initializes a logger (
-
Configuration:
- Loads application configuration using
FileHandler
from a file typically namedapp_id.config
in theconfig_dir
. - Defines default configuration
keys
anddefaults
(e.g., for macros, helpers, debug status).
- Loads application configuration using
-
Core Attributes:
version
: ToolBoxV2 version, read frompyproject.toml
.debug
: Boolean, controlled by CLI args and config.dev_modi
: Boolean, development mode status from config.functions
: A dictionary to store registered functions from modules.modules
: A dictionary to store loaded module objects.interface_type
: DefaultToolBoxInterfaces.native
.alive
: Boolean, controls the main application loop.args_sto
: Stores the parsedAppArgs
.loop
: The asyncio event loop (initialized later or if already running).session
: ASession
object for managing user/remote session state.
-
Conditional Actions (based on
AppArgs
):args.init
: If true, addsstart_dir
tosys.path
.- The
__main__.py
script handles other arguments like--update
,--get-version
, etc., by callingApp
methods or other utilities.
Core Functionalities¶
Module Management¶
-
load_mod(mod_name: str, spec='app', mlm='I', **kwargs)
/save_load(modname, spec='app')
:- Loads a module into the application.
spec
(specification): Used to namespace or categorize the module instance (e.g., 'app' for general, or a specific session ID).- Supports different loading mechanisms (
mlm
):'I'
: In-place load (imports the Python module directly). This is the default.'C'
: Copies the module file to a runtime directory before loading (less common).
- Handles
ModuleNotFoundError
by attempting to guide the user (e.g., install viaCloudM
orpip
). - Registers the module's exported functions and/or its
Tools
class instance. - Can reload modules if they are already loaded.
1 2 3 4 5
# Load the 'MyModule' my_module_instance = app_instance.load_mod("MyModule") # Or if it's a Tool-based module: # my_tool_instance = app_instance.load_mod("MyToolModule")
-
get_mod(name: str, spec='app') -> ModuleType | MainToolType
:- Retrieves a loaded module instance. If the module isn't loaded, it attempts to load it.
1 2 3
db_mod = app_instance.get_mod("DB") if db_mod: db_mod.some_db_function()
- Retrieves a loaded module instance. If the module isn't loaded, it attempts to load it.
-
remove_mod(mod_name: str, spec='app', delete=True)
/a_remove_mod(...)
(async):- Unloads a module, calling its
on_exit
functions if defined. delete=True
removes it completely from thefunctions
registry.
- Unloads a module, calling its
-
reload_mod(mod_name: str, spec='app', ...)
:- Reloads an existing module. Useful for development.
-
watch_mod(mod_name: str, ...)
:- Monitors a module's source file(s) for changes and automatically reloads it.
1 2
# In development, watch 'MyDevModule' for changes app_instance.watch_mod("MyDevModule")
- Monitors a module's source file(s) for changes and automatically reloads it.
-
load_all_mods_in_file(working_dir="mods")
/a_load_all_mods_in_file(...)
(async):- Scans the specified directory (default
./mods/
) and loads all valid Python modules found.
- Scans the specified directory (default
To isaa¶
Function Registration and Execution¶
-
@app.tb(...)
Decorator (via_create_decorator
):- The primary way functions are registered with ToolBoxV2. See
example_mod.md
for details on usage. - This decorator populates the
app.functions
dictionary.
- The primary way functions are registered with ToolBoxV2. See
-
get_function(name: Enum | tuple, metadata=False, state=True, specification='app', ...)
:- Retrieves a registered function.
name
: Can be an Enum (fromall_functions_enums.py
) or a(module_name, function_name)
tuple.metadata=True
: Returns a tuple(function_data_dict, callable_function)
.state=True
: Returns a stateful version of the function (bound to its module instance if applicable).state=False
: Returns the raw, stateless function.specification
: The context/instance spec to get the function for.
-
run_any(mod_function_name, ..., get_results=False, **kwargs)
/a_run_any(...)
(async):- Executes a registered function by its name (Enum or tuple).
- Handles argument passing, stateful/stateless execution, and error wrapping into a
Result
object. get_results=True
: Returns theResult
object itself.get_results=False
(default): Returns thedata
payload from theResult
object if successful.- Automatically handles running the function's pre/post compute hooks and caching if configured via
@app.tb
.1 2 3 4 5 6
# Synchronous execution result_data = app_instance.run_any(("MyModule", "my_function"), arg1="hello") full_result_obj = app_instance.run_any(("MyModule", "my_function"), arg1="hello", get_results=True) # Asynchronous execution async_result_data = await app_instance.a_run_any(("MyAsyncModule", "my_async_function"))
-
run_http(mod_function_name, function_name=None, method="GET", ...)
(async):- Executes a function on a remote ToolBoxV2 instance via HTTP, using the app's
session
object.
- Executes a function on a remote ToolBoxV2 instance via HTTP, using the app's
Application Lifecycle¶
exit()
/a_exit()
(async):- Gracefully shuts down the application.
- Calls
on_exit
functions for all loaded modules. - Saves configuration.
- Stops the main application loop (
alive = False
). - Cleans up threads and the event loop if applicable.
Utilities¶
print(text, *args, **kwargs)
/sprint(text, *args, **kwargs)
:- Styled print functions, prepending
System$[app_id]:
.sprint
is often used for more verbose/system-level messages and can be silenced.
- Styled print functions, prepending
debug_rains(e: Exception)
: Ifapp.debug
is true, prints a full traceback and re-raises the exception.set_flows(r: dict)
/run_flows(name: str, **kwargs)
: Manages and executes predefined application flows (sequences of operations).get_username()
/set_username(username: str)
: Manages the application's user identity.save_autocompletion_dict()
/get_autocompletion_dict()
: Saves/loads a dictionary of modules and their functions for autocompletion features.save_registry_as_enums(directory: str, filename: str)
: Generates anall_functions_enums.py
-like file from the currently registered functions.execute_all_functions(...)
/a_execute_all_functions(...)
(async):- Runs all registered testable functions (marked with
test=True
in@app.tb
or havingsamples
). - Useful for integration testing and profiling.
- Can filter by module (
m_query
) and function (f_query
). - Supports profiling via
cProfile
.
- Runs all registered testable functions (marked with
run_bg_task(task: Callable)
:- Runs a synchronous or asynchronous task in a separate background thread with its own event loop. Ensures proper handling of nested asyncio operations within the task.
Session Management (app.session
)¶
The app.session
attribute holds an instance of the Session
class (from toolboxv2.utils.system.session
). It's used for:
* Authenticating with a remote ToolBoxV2 service (e.g., SimpleCore Hub).
* Making authenticated HTTP requests (session.fetch
, session.upload_file
, session.download_file
).
* Manages JWT claims and private key authentication.
1 2 3 4 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
The CLI script (__main__.py
) performs the following main steps:
1. Parses command-line arguments.
2. Initializes the App
instance via setup_app()
(which calls get_app()
).
3. Handles various options to:
* Manage application data and configuration.
* Control application modes (background, proxy, debug).
* Load modules and manage their state.
* Run tests or profilers.
* Execute specific application flows or commands.
Key CLI Arguments¶
The following are some of the primary arguments available. Use tb -h
or python -m toolboxv2 -h
for a full list.
-
Instance and Mode:
-init [name]
: Initializes ToolBoxV2 with a specific instance name (default:main
).-n, --name <name>
: Specifies an ID for the ToolBox instance (default:main
). This affects data/config directory names.-m, --modi <mode>
: Starts a ToolBoxV2 interface/flow (e.g.,cli
,bg
, or custom flows). Default is usually "cli".--debug
: Starts the application in debug mode (more verbose logging, potentially different behavior).--remote
: Starts in remote mode, often for connecting to a proxy or another instance.-bg, --background-application
: Starts an interface in the background as a detached process.-bgr, --background-application-runner
: Runs the background application logic in the current terminal (for daemons).-fg, --live-application
: Starts a proxy interface, connecting to a background daemon.--kill
: Kills the currently running local ToolBoxV2 instance (matching the-m <mode>
and-n <name>
).
-
Module and Version Management:
-l, --load-all-mod-in-files
: Loads all modules found in themods/
directory on startup.-sfe, --save-function-enums-in-file
: Generates/updates theall_functions_enums.py
file based on loaded modules. Often used with-l
.-v, --get-version
: Prints the version of ToolBoxV2 and all loaded modules.-i, --install <name>
: Installs a module or interface (likely viaCloudM
module).-r, --remove <name>
: Uninstalls a module or interface.-u, --update <name_or_main>
: Updates a module/interface or the core ToolBoxV2 (main
).
-
Development and Testing:
--test
: Runs all unit tests (typically discovers and runs tests in thetests/
directory).--profiler
: Runs all registered testable functions and profiles their execution usingcProfile
.--ipy
: Starts an IPython session with the ToolBoxV2 app pre-loaded. Provides magic commands like%tb save/loadX/load/open/inject
.
-
Service Management (
--sm
):- Provides a sub-menu for managing ToolBoxV2 as a system service (Linux/systemd or Windows Startup).
- Options: Init, Start/Stop/Restart, Status, Uninstall, Show/Hide console window (Windows).
-
Log Management (
--lm
):- Provides a sub-menu for managing log files (e.g., removing or unstyling logs by date/level).
-
Data and Configuration Management:
--delete-config-all
: Deletes all configuration files. Use with caution!--delete-data-all
: Deletes all data folders. Use with caution!--delete-config
: Deletes the configuration file for the named instance.--delete-data
: Deletes the data folder for the named instance.
-
Network Configuration (for interfaces):
-p, --port <port>
: Specifies the port for an interface (default:5000
or6587
for background).-w, --host <host>
: Specifies the host for an interface (default:0.0.0.0
).
-
Direct Command Execution:
-c, --command <module_name> <function_name> [arg1 arg2 ...]
(can be repeated): Executes a specific function.--kwargs <key=value>
(can be repeated): Provides keyword arguments for commands specified with-c
.
-
Conda Integration:
conda [conda_args...]
: Special argument to pass commands directly to aconda_runner.py
script (e.g.,tb conda env list
).
-
API Runner:
api [api_args...]
: Special argument to invokecli_api_runner.py
, likely for direct API interactions or testing.
-
GUI:
gui
: Starts the GUI version of ToolBoxV2 (imports and runstoolboxv2.__gui__.start
).
CLI Execution Flow (__main__.py
)¶
- Argument Parsing:
parse_args()
usesargparse
to define and parse all CLI arguments. - App Setup (
setup_app()
):- Initializes the
App
instance usingget_app()
with the parsed arguments and name. - Sets up PID file for the current process.
- Optionally silences
app.sprint
if not in debug/verbose mode. - Loads all modules if
-l
is specified. - Handles
--update
logic.
- Initializes the
- Background/Live Application Handling:
- If
-bgr
: InitializesDaemonApp
. - If
-bg
: Starts the application as a detached background process usingsubprocess.Popen
. - If
-fg
(live-application): Attempts to connect to a background daemon usingProxyApp
.
- If
- Action Dispatching: Based on the parsed arguments, it performs actions like:
- Module installation (
--install
). - Log management (
--lm
). - Service management (
--sm
). - Saving function enums (
-sfe
). - Printing versions (
-v
). - Running the profiler (
--profiler
). - Running flows based on
--modi
. - Handling Docker commands (
--docker
). - Killing an existing instance (
--kill
). - Executing direct commands (
-c
).
- Module installation (
- Cleanup: Removes the PID file and calls
app.a_exit()
before exiting.
Example CLI Usage¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 |
|
@export
Decorator Parameters:¶
name
(str, optional): The name to register the function under. Defaults to the function's actual name.mod_name
(str): The name of the module this function belongs to. If not provided, it tries to infer fromfunc.__module__
or a globalName
in the module.version
(str, optional): Version string for this function/feature. Combined with the app's version.helper
(str, optional): A docstring or help text for the function.api
(bool, defaultFalse
): IfTrue
, exposes this function as an HTTP API endpoint.- The URL pattern is typically
/api/<mod_name>/<func_name>
. - For streaming,
/sse/<mod_name>/<func_name>
.
- The URL pattern is typically
api_methods
(List[str], optional): Specifies allowed HTTP methods (e.g.,['GET', 'POST']
). Defaults to['AUTO']
(GET if no body params, POST if body params).request_as_kwarg
(bool, defaultFalse
): IfTrue
andapi=True
, the function will receive arequest: RequestData
keyword argument if it's defined in its signature.row
(bool, defaultFalse
): IfTrue
, the function's raw return value is used directly. IfFalse
(default), the return value is automatically wrapped in aResult.ok()
object if it's not already aResult
orApiResult
.initial
(bool, defaultFalse
): IfTrue
, this function is added to the module's "on_start" list and is called when the module is loaded by theApp
instance (if the module instance is aMainTool
or similar, or if called directly).exit_f
(bool, defaultFalse
): IfTrue
, this function is added to the module's "on_exit" list and is called when theApp
instance is shutting down or the module is removed.state
(bool, optional):- If
None
(default): Automatically determined. If the first parameter of the decorated function is namedself
orapp
(and type-hinted asApp
),state
is consideredTrue
. OtherwiseFalse
. - If
True
: The function is considered stateful. If its first parameter isself
, it's assumed to be a method of a class instance (e.g., aMainTool
subclass). Ifapp
, theApp
instance is passed. - If
False
: The function is treated as stateless.
- If
test
(bool, defaultTrue
): Marks the function as testable. Used byapp.execute_all_functions()
.samples
(List[Dict[str, Any]], optional): A list of sample keyword arguments to be used when testing the function withapp.execute_all_functions()
.memory_cache
(bool, defaultFalse
): Enables in-memory caching for the function's results.memory_cache_ttl
(int, default300
): Time-to-live in seconds for memory cache entries.memory_cache_max_size
(int, default100
): Max number of entries in the memory cache.file_cache
(bool, defaultFalse
): Enables file-based caching for the function's results. (Stored inapp.data_dir/cache/...
).restrict_in_virtual_mode
(bool, defaultFalse
): IfTrue
, restricts function in certain virtualized/proxied modes.level
(int, default-1
): A general-purpose level or priority for the function.pre_compute
(Callable, optional): A function(func, *args, **kwargs) -> (args, kwargs)
called before the main function executes. It can modify args/kwargs.post_compute
(Callable, optional): A function(result, func, *args, **kwargs) -> result
called after the main function executes. It can modify the result.interface
(ToolBoxInterfaces | str, optional): Specifies the intended interface type (e.g.,ToolBoxInterfaces.cli
,ToolBoxInterfaces.api
). Defaults to "tb".
Result
and ApiResult
Objects¶
- Modules should typically return
Result
objects (orApiResult
for API endpoints) to provide standardized responses including success/error status, data payload, and informational messages. - The
toolboxv2.utils.system.types.Result
class provides factory methods:Result.ok(data=..., info=...)
Result.json(data=..., info=...)
(forapi=True
functions)Result.html(data=..., info=...)
Result.text(text_data=..., info=...)
Result.binary(data=..., content_type=..., download_name=...)
Result.redirect(url=..., status_code=...)
Result.stream(stream_generator=..., info=..., cleanup_func=...)
(for SSE)Result.default_user_error(info=..., exec_code=...)
Result.default_internal_error(info=..., exec_code=...)
Result.custom_error(data=..., info=..., exec_code=...)
- The
Result
object has atask(background_task_callable)
method to schedule a background task to run after the main function returns.
RequestData
Object¶
- For API functions (
api=True
) withrequest_as_kwarg=True
, if the function signature includesrequest: Optional[RequestData] = None
, it will receive an instance oftoolboxv2.utils.system.types.RequestData
. RequestData
provides access to:request.method
,request.path
request.headers
(an instance ofHeaders
, e.g.,request.headers.user_agent
,request.headers.hx_request
)request.query_params
(dict)request.form_data
(dict, if applicable)request.body
(parsed JSON ifcontent_type
isapplication/json
, otherwise raw bytes/str)session.SiID
,session.user_name
,session.level
(from the current user's session)
Creating a MainTool
-based Module¶
For more complex, stateful modules, you can create a class that inherits from toolboxv2.utils.system.main_tool.MainTool
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
|
Key aspects of MainTool
:
* Asynchronous Initialization: Use async def __ainit__(self)
for setup. The MainTool
itself is awaitable, and __ainit__
is called when the instance is first awaited (e.g., by app.load_mod
or app.get_mod
).
* self.app
: The App
instance is available as self.app
.
* self.name
, self.version
, self.logger
: These are automatically set up.
* self.return_result(...)
: A helper method for creating Result
objects.
* Methods intended to be called via app.run_any
should be decorated with @export
.
Steps to Create a Valid Toolboxv2 Module:¶
- Define Module Structure: Organize your code with imports, metadata, and function/class definitions.
- Clarify Dependencies: Import necessary libraries. Handle missing optional dependencies gracefully if needed.
- Export Functions/Methods: Use the
@export(...)
decorator (e.g.,app.tb(...)
) to mark functions/methods that ToolBoxV2 should recognize.- Provide
mod_name
andversion
. - Use other parameters (
api
,row
,initial
,exit_f
,memory_cache
, etc.) as needed. - Ensure clear signatures and document parameters/return types (Python type hints are highly recommended).
- Provide
- Documentation and Versioning: Document your module and its functions. Use semantic versioning.
- Testing: Test your module thoroughly, including how it integrates with the ToolBoxV2 app (
app.run_any
,app.get_mod
, etc.). Use thetest=True
andsamples
parameters in@export
to facilitate automated testing viaapp.execute_all_functions()
.