Commit Graph

25 Commits

Author SHA1 Message Date
Maarten van der Schrieck 3bca93b2d0 ports: Fix sys.stdout.buffer.write() return value.
MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.

One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.

A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:

mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.

The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
  multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes

As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().

Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().

This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.

Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
2023-12-22 10:32:46 +11:00
Jim Mussared 5015779a6f py/builtinevex: Handle invalid filenames for execfile.
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.

This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).

This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.

Fixes issue #12522.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-12 15:17:59 +11:00
Jim Mussared 79473691f2 {mimxrt,powerpc,samd}/mpconfigport: Don't override parse chunk alloc.
This was copied from minimal/mpconfigport.h, but it doesn't make sense
for general ports.

Add a comment to minimal/mpconfigport.h to explain why it specifically
overrides it.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-29 14:11:26 +10:00
Damien George b1229efbd1 all: Fix spelling mistakes based on codespell check.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
David Grayson c046b23ea2 shared/runtime/pyexec: Don't allow Ctrl+C to interrupt frozen boot code.
Helps prevent the filesystem from getting formatted by mistake, among other
things.  For example, on a Pico board, entering Ctrl+D and Ctrl+C fast many
times will eventually wipe the filesystem (without warning or notice).

Further rationale: Ctrl+C is used a lot by automation scripts (eg mpremote)
and UI's (eg Mu, Thonny) to get the board into a known state.  If the board
is not responding for a short time then it's not possible to know if it's
just a slow start up (eg in _boot.py), or an infinite loop in the main
application.  The former should not be interrupted, but the latter should.
The only way to distinguish these two cases would be to wait "long enough",
and if there's nothing on the serial after "long enough" then assume it's
running the application and Ctrl+C should break out of it.  But defining
"long enough" is impossible for all the different boards and their possible
behaviour.  The solution in this commit is to make it so that frozen
start-up code cannot be interrupted by Ctrl+C.  That code then effectively
acts like normal C start-up code, which also cannot be interrupted.

Note: on the stm32 port this was never seen as an issue because all
start-up code is in C.  But now other ports start to put more things in
_boot.py and so this problem crops up.

Signed-off-by: David Grayson <davidegrayson@gmail.com>
2023-04-05 10:38:50 +10:00
Damien George b3c8ab37ec py/gc: Make gc_dump_info/gc_dump_alloc_table take a printer as argument.
So that callers can redirect the output if needed.

Signed-off-by: Damien George <damien@micropython.org>
2023-03-10 10:58:10 +11:00
Jim Mussared 17f2783e4a all: Use += rather than = everywhere for CFLAGS/LDFLAGS/LIBS.
This avoids a surprise where an = can cancel out an earlier +=.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:17:41 +11:00
Jim Mussared 6ecdf1a240 tests/frozen: Move frozentest.mpy from ports/ to tests/.
frozentest.mpy was previously duplicated in ports/minimal and
ports/powerpc.

This needs to be re-generated on every .mpy version increase, so might as
well just have a single copy of it.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 23:51:10 +10:00
David Lechner 68f46342aa shared/runtime/pyexec: Use MP_REGISTER_ROOT_POINTER().
This uses MP_REGISTER_ROOT_POINTER() to register repl_line
instead of using a conditional inside of mp_state_vm_t.

Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18 13:52:01 +10:00
David Lechner 81dbea1ce3 shared/readline: Use MP_REGISTER_ROOT_POINTER().
This uses MP_REGISTER_ROOT_POINTER() to register the readline_history root
pointer array used by shared/readline.c and removes the registration from
all mpconfigport.h files.

This also required adding a new MICROPY_READLINE_HISTORY_SIZE config option
since not all ports used the same sized array.

Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18 13:48:49 +10:00
David Lechner c118b5d0e4 extmod/extmod.mk: Separate out extmod file list from py.mk to extmod.mk.
This separates extmod source files from `py.mk`.  Previously, `py.mk`
assumed that every consumer of the py/ directory also wanted to include
extmod/.  However, this is not the case.  For example, building mpy-cross
uses py/ but doesn't need extmod/.

This commit moves all extmod-specific items from `py.mk` to `extmod.mk` and
explicitly includes `extmod.mk` in ports that use it.

Signed-off-by: David Lechner <david@pybricks.com>
2022-06-21 00:14:34 +10:00
Damien George 5956466c0e py/builtin: Clean up and simplify import_stat and builtin_open config.
The following changes are made:

- If MICROPY_VFS is enabled then mp_vfs_import_stat and mp_vfs_open are
  automatically used for mp_import_stat and mp_builtin_open respectively.

- If MICROPY_PY_IO is enabled then "open" is automatically included in the
  set of builtins, and points to mp_builtin_open_obj.

This helps to clean up and simplify the most common port configuration.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-25 13:04:45 +10:00
Damien George 24894f9065 ports: Recompile bytecode tests now that .mpy format changed.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George c0f2af4e86 ports: Recompile bytecode tests now that .mpy format changed.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-24 18:29:02 +11:00
Damien George 136369d72f all: Update to point to files in new shared/ directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 17:08:10 +10:00
Damien George 7c44354592 ports: Remove def of MP_PLAT_PRINT_STRN if it's the same as the default.
To simplify config, there's no need to specify MP_PLAT_PRINT_STRN if it's
the same as the default definition in py/mpconfig.h.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:39:17 +11:00
Joel Stanley f03d030080 powerpc/uart: Choose which UART to use at build time, not runtime.
Microwatt may have firmware that places data in r3, which was used to
detect microwatt vs powernv.  This breaks the existing probing of the UART
type in this powerpc port.

Instead build only the appropriate UART into the firmware, selected by
passing the option UART=potato or UART=lpc_serial to the Makefile.

A future enhancement would be to parse the device tree and configure
MicroPython based on the settings.
2020-05-29 22:54:55 +10:00
Joel Stanley b65482ffa8 powerpc: Set better default compiler.
Most developers use a compiler which is called powerpc64le-linux-gnu-gcc.
2020-05-27 17:02:04 +10:00
Joel Stanley 25bc42e754 powerpc: Fix Makefile rule when linking.
The linker script was included in the "$^" inputs, causing the build to
fail:

 LINK build/firmware.elf
 powerpc64le-linux-gnu-ld: error: linker script file 'powerpc.lds' appears multiple times

As a fix the linker script is left as a dependency of the elf, but only the
object files are linked.
2020-05-27 17:00:44 +10:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
stijn 30840ebc99 all: Enable extra conversion warnings where applicable.
Add -Wdouble-promotion and -Wfloat-conversion for most ports to ban out
implicit floating point conversions, and add extra Travis builds using
MICROPY_FLOAT_IMPL_FLOAT to uncover warnings which weren't found
previously.  For the unix port -Wsign-comparison is added as well but only
there since only clang supports this but gcc doesn't.
2020-04-18 22:42:28 +10:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 3f39d18c2b all: Add *FORMAT-OFF* in various places.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments.  This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...).  In other places
it is used to prevent excessive reformatting that would make the code less
readable.
2020-02-28 10:31:07 +11:00
David Lechner e79424d672 ports: Allow overriding CROSS_COMPILE in a custom makefile.
Many ports already allow overriding CROSS_COMPILE.  This modifies the
remaining ports to allow it as well.
2019-12-27 23:53:16 +11:00
Michael Neuling 079cc940a6 powerpc: Add initial port to bare metal PowerPC arch.
Runs in microwatt (GHDL and FPGA) and qemu.

Port done initially by Michael Neuling, with help from Anton Blanchard and
Jordan Niethe.
2019-10-22 22:45:33 +11:00