Commit Graph

173 Commits

Author SHA1 Message Date
Maarten van der Schrieck 3bca93b2d0 ports: Fix sys.stdout.buffer.write() return value.
MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.

One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.

A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:

mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.

The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
  multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes

As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().

Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().

This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.

Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
2023-12-22 10:32:46 +11:00
Damien George ad806df857 ports: Move definitions of ATOMIC_SECTION macros to mphalport.h.
Also move MICROPY_PY_PENDSV_ENTER/REENTER/EXIT to mphalport.h, for ports
where these are not already there.

This helps separate the hardware implementation of these macros from the
MicroPython configuration (eg for renesas-ra and stm32, the IRQ static
inline helper functions can now be moved to irq.h).

Signed-off-by: Damien George <damien@micropython.org>
2023-12-01 14:37:48 +11:00
Damien George e1ec6af654 extmod/modmachine: Provide common bindings for 6 bare-metal functions.
Minor changes for consistency are:
- nrf gains: unique_id(), freq() [they do nothing]
- samd: deepsleep() now resets after calling lightsleep()
- esp32: lightsleep()/deepsleep() no longer take kw arg "sleep", instead
  it's positional to match others.  also, passing 0 here will now do a 0ms
  sleep instead of acting like nothing was passed.
  reset_cause() no longer takes any args (before it would just ignore them)
- mimxrt: freq() with an argument and lightsleep() both raise
  NotImplementedError

Signed-off-by: Damien George <damien@micropython.org>
2023-11-30 16:11:11 +11:00
Damien George 30a63a204d extmod/modmachine: Provide common Python bindings for machine.idle().
And use it in all ports.  The ports are unchanged, except esp8266 which now
just returns None from this function instead of the time elapsed (to match
other ports), and qemu-arm which gains this function.

Signed-off-by: Damien George <damien@micropython.org>
2023-11-30 16:11:11 +11:00
Damien George 7d39db2503 extmod/modmachine: Factor ports' machine module dict to common code.
This is a code factoring to have the dict for the machine module in one
location, and all the ports use that same dict.  The machine.soft_reset()
function implementation is also factored because it's the same for all
ports that did already implement it.  Eventually more functions/bindings
can be factored.

All ports remain functionally the same, except:
- cc3200 port: gains soft_reset, mem8, mem16, mem32, Signal; loses POWER_ON
  (which was a legacy constant, replaced long ago by PWRON_RESET)
- nrf port: gains Signal
- qemu-arm port: gains soft_reset
- unix port: gains soft_reset
- zephyr port: gains soft_reset, mem8, mem16, mem32

Signed-off-by: Damien George <damien@micropython.org>
2023-11-30 16:11:11 +11:00
Damien George 3e2706a18d extmod/modmachine: Consolidate mem, i2c and spi headers to modmachine.h.
The contents of machine_mem.h, machine_i2c.h and machine_spi.h have been
moved into extmod/modmachine.h.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-26 17:40:22 +11:00
Damien George d336c1b79b extmod/modmachine: Consolidate simple machine headers into modmachine.h.
The contents of machine_bitstream.h, machine_pinbase.h, machine_pulse.h and
machine_signal.h have been moved into extmod/modmachine.h.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-26 16:49:27 +11:00
Damien George 90023b4dcf extmod/modmachine: Clean up decls of machine types to use common ones.
The machine_i2c_type, machine_spi_type and machine_timer_type symbols are
already declared in extmod/modmachine.h and should not be declared anywhere
else.

Also move declarations of machine_pin_type and machine_rtc_type to the
common header in extmod.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-26 16:20:53 +11:00
Damien George 5b4a2baff6 extmod/machine_uart: Factor ports' UART Python bindings to common code.
This is a code factoring to have the Python bindings in one location, and
all the ports use those same bindings.  For all ports except the two listed
below there is no functional change.

The nrf port has UART.sendbreak() removed, but this method previously did
nothing.

The zephyr port has the following methods added:
- UART.init(): supports setting timeout and timeout_char.
- UART.deinit(): does nothing, just returns None.
- UART.flush(): raises OSError(EINVAL) because it's not implemented.
- UART.any() and UART.txdone(): raise NotImplementedError.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-26 10:46:42 +11:00
Jim Mussared 5015779a6f py/builtinevex: Handle invalid filenames for execfile.
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.

This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).

This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.

Fixes issue #12522.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-12 15:17:59 +11:00
Damien George f01d5fb657 py/mkrules.mk: Automatically configure frozen options when manifest set.
Following how mkrules.cmake works.  This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 23:12:56 +10:00
Jim Mussared 2eba98f1e0 all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:21 +10:00
Jim Mussared 45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
Jim Mussared 1bf2dcb15e all: Rename mp_umodule*, mp_module_umodule* to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:04 +10:00
Jim Mussared dfe232d000 py/builtinimport: Remove weak links.
In order to keep "import umodule" working, the existing mechanism is
replaced with a simple fallback to drop the "u".

This makes importing of built-ins no longer touch the filesystem, which
makes a typical built-in import take ~0.15ms rather than 3-5ms.

(Weak links were added in c14a81662c)

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:04 +10:00
Jim Mussared 30628d1bb7 all: Rename MP_QSTR_umodule to MP_QSTR_module everywhere.
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:53:57 +10:00
Jim Mussared 1d4b4f0ce2 ports: Standardise docs link in help text.
Updates all `help()` output to use the phrase:
`For online docs please visit http://docs.micropython.org/`

Some ports previously used different wording, some pointed to the wrong
link.  Also make all ports use `help.c` for consistency.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 11:48:46 +10:00
Damien George 143b863f54 zephyr/modutime: Use extmod version of time module.
API change: time.time_ns() is added, but it just returns 0.

No API or functional change to existing time functions.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:11:52 +10:00
Damien George 9955553001 extmod/modutime: Provide a generic time module.
Based on extmod/utime_mphal.c, with:
- a globals dict added
- time.localtime wrapper added
- time.time wrapper added
- time.time_ns function added

New configuration options are added for this module:
- MICROPY_PY_UTIME (enabled at basic features level)
- MICROPY_PY_UTIME_GMTIME_LOCALTIME_MKTIME
- MICROPY_PY_UTIME_TIME_TIME_NS

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:09:56 +10:00
Jim Mussared bc9ec1cf71 extmod/modbluetooth: Merge gatts_notify/indicate implementation.
Makes gatts_notify and gatts_indicate work in the same way: by default they
send the DB value, but you can manually override the payload.

In other words, makes gatts_indicate work the same as gatts_notify.

Note: This removes support for queuing notifications and indications on
btstack when the ACL buffer is full. This functionality will be
reimplemented in a future commit.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Damien George b3c8ab37ec py/gc: Make gc_dump_info/gc_dump_alloc_table take a printer as argument.
So that callers can redirect the output if needed.

Signed-off-by: Damien George <damien@micropython.org>
2023-03-10 10:58:10 +11:00
iabdalkader 5e90ec2331 zephyr: Rename machine I2C and SPI types consistently across ports.
This renames:
- machine_hard_i2c_type -> machine_i2c_type
- machine_hard_spi_type -> machine_spi_type
2022-10-22 12:58:21 +11:00
Jim Mussared 94beeabd2e py/obj: Convert make_new into a mp_obj_type_t slot.
Instead of being an explicit field, it's now a slot like all the other
methods.

This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:15 +10:00
Jim Mussared 6da41b5900 py/obj: Merge getiter and iternext mp_obj_type_t slots.
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.

This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
  type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
  to a mp_getiter_iternext_custom_t instance, which then defines both
  getiter and iternext.

And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.

Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared 9dce82776d all: Remove unnecessary locals_dict cast.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared 662b9761b3 all: Make all mp_obj_type_t defs use MP_DEFINE_CONST_OBJ_TYPE.
In preparation for upcoming rework of mp_obj_type_t layout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Andrew Leech afc7e1d298 zephyr/mpconfigport: Enable MICROPY_ENABLE_FINALISER when VFS is used. 2022-09-13 13:08:03 +10:00
Damien George 1d4228ccc1 zephyr/boards: Add config for bbc_microbit_v2.
This enables the bluetooth module.  GAP scanning and advertising works.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-09 14:14:31 +10:00
Jim Mussared 8a0ee5a5c0 py/objstr: Split mp_obj_str_from_vstr into bytes/str versions.
Previously the desired output type was specified.  Now make the type part
of the function name.  Because this function is used in a few places this
saves code size due to smaller call-site.

This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c
(which is almost the only place where the output type isn't a compile-time
constant).

This saves ~140 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:43:55 +10:00
Jim Mussared 28aaab9590 py/objstr: Add hex/fromhex to bytes/memoryview/bytearray.
These were added in Python 3.5.

Enabled via MICROPY_PY_BUILTINS_BYTES_HEX, and enabled by default for all
ports that currently have ubinascii.

Rework ubinascii to use the implementation of these methods.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-12 12:44:30 +10:00
Damien George 6c67fbc280 zephyr/machine_uart: Use mp_obj_str_get_str to get device name.
This checks that the argument is actually a string.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-11 23:18:02 +10:00
David Lechner ccda7686a4 zephyr: Use MP_REGISTER_ROOT_POINTER().
This uses MP_REGISTER_ROOT_POINTER() to register port-specific root
pointers for the zephyr port.

Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18 13:50:35 +10:00
David Lechner 81dbea1ce3 shared/readline: Use MP_REGISTER_ROOT_POINTER().
This uses MP_REGISTER_ROOT_POINTER() to register the readline_history root
pointer array used by shared/readline.c and removes the registration from
all mpconfigport.h files.

This also required adding a new MICROPY_READLINE_HISTORY_SIZE config option
since not all ports used the same sized array.

Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18 13:48:49 +10:00
Maureen Helm bada8b3cad zephyr: Upgrade to Zephyr v3.1.0.
Updates the Zephyr port build instructions and CI to use the latest Zephyr
release tag.

Tested on frdm_k64f.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-06-17 16:51:34 +10:00
Maureen Helm 4fd54a4756 zephyr: Update include paths to use the zephyr namespace.
Zephyr v3.1.0 moved all public headers to include/zephyr.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-06-17 16:44:30 +10:00
Damien George d75892c0b0 zephyr/modusocket: Fix function object wrapper for listen method.
This was missed by 919f696ad2

Signed-off-by: Damien George <damien@micropython.org>
2022-06-15 11:38:09 +10:00
Damien George efe23aca71 all: Remove third argument to MP_REGISTER_MODULE.
It's no longer needed because this macro is now processed after
preprocessing the source code via cpp (in the qstr extraction stage), which
means unused MP_REGISTER_MODULE's are filtered out by the preprocessor.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-02 16:31:37 +10:00
Damien George 6e71cde6aa ports: Use default VFS config for import_stat and builtin_open.
For ports with MICROPY_VFS and MICROPY_PY_IO enabled their configuration
can now be simplified to use the defaults for mp_import_stat and
mp_builtin_open.

This commit makes no functional change, except for the following minor
points:
- the built-in "open" is removed from the minimal port (it previously did
  nothing)
- the duplicate built-in "input" is removed from the esp32 port
- qemu-arm now delegates to VFS import/open

Signed-off-by: Damien George <damien@micropython.org>
2022-05-25 13:04:45 +10:00
Damien George 5956466c0e py/builtin: Clean up and simplify import_stat and builtin_open config.
The following changes are made:

- If MICROPY_VFS is enabled then mp_vfs_import_stat and mp_vfs_open are
  automatically used for mp_import_stat and mp_builtin_open respectively.

- If MICROPY_PY_IO is enabled then "open" is automatically included in the
  set of builtins, and points to mp_builtin_open_obj.

This helps to clean up and simplify the most common port configuration.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-25 13:04:45 +10:00
Jim Mussared ec2fe5d6ca zephyr: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared 4eab44a1ec extmod: Make extmod modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Jim Mussared 0e7bfc88c6 all: Use mp_obj_malloc everywhere it's applicable.
This replaces occurences of

    foo_t *foo = m_new_obj(foo_t);
    foo->base.type = &foo_type;

with

    foo_t *foo = mp_obj_malloc(foo_t, &foo_type);

Excludes any places where base is a sub-field or when new0/memset is used.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:28:14 +10:00
Damien George 5c32111fa0 ports: Reformat more C and Python source code.
These files that are reformatted only now fall under the list of files to
apply uncrustify/black formatting to.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-20 16:17:35 +10:00
Damien George aab005c75b extmod/modusocket: Provide config macro for socket.listen backlog deflt.
To make it possible to change this value for any given port or board.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-11 15:28:56 +10:00
Jon Bjarni Bjarnason 919f696ad2 extmod/modusocket: Implement optional socket.listen backlog argument.
This follows the CPython change: https://bugs.python.org/issue21455

Socket listen backlog defaults to 2 if not given, based on most bare metal
targets not having many resources for a large backlog.  On UNIX it defaults
to SOMAXCONN or 128, whichever is less.
2022-04-11 15:26:47 +10:00
Damien George 818be10bb5 zephyr/moduos: Convert module to use extmod version.
This also adds uos.unlink(), for all ports that use extmod/moduos.c.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Damien George 926b554daf extmod/moduos: Create general uos module to be used by all ports.
Based on the rp2 port version, with the rp2 port converted to use this
module.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Damien George cced9a0128 zephyr/prj.conf: Enable CONFIG_BUILD_OUTPUT_HEX to generate .hex output.
The .hex file contains more information than .bin, useful for flashing.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-07 16:48:35 +11:00
Maureen Helm 474d288e55 zephyr: Upgrade to Zephyr v3.0.0.
Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-03-07 16:48:35 +11:00