Commit Graph

350 Commits

Author SHA1 Message Date
Jim Mussared 74fd7b3d32 tools/ci.sh: Set `ulimit -n` for unix CI.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-12-21 11:05:31 +11:00
Jim Mussared 8b24aa36ba extmod/modselect: Handle growing the pollfds allocation correctly.
The poll_obj_t instances have their pollfd field point into this
allocation.  So if re-allocating results in a move, we need to update the
existing poll_obj_t's.

Update the test to cover this case.

Fixes issue #12887.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-12-21 11:04:59 +11:00
Carlosgg f3f215e9bd extmod/modssl_mbedtls: Add SSLContext certificate methods.
This commit adds:

1) Methods to SSLContext class that match CPython signature:

	- `SSLContext.load_cert_chain(certfile, keyfile)`
	- `SSLContext.load_verify_locations(cafile=, cadata=)`
	- `SSLContext.get_ciphers()` --> ["CIPHERSUITE"]
	- `SSLContext.set_ciphers(["CIPHERSUITE"])`

2) `sslsocket.cipher()` to get current ciphersuite and protocol
   version.

3) `ssl.MBEDTLS_VERSION` string constant.

4) Certificate verification errors info instead of
   `MBEDTLS_ERR_X509_CERT_VERIFY_FAILED`.

5) Tests in `net_inet` and `multi_net` to test these new methods.

`SSLContext.load_cert_chain` method allows loading key and cert from disk
passing a filepath in `certfile` or `keyfile` options.

`SSLContext.load_verify_locations`'s `cafile` option enables the same
functionality for ca files.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-12-12 16:25:07 +11:00
Damien George c554df57f6 tests/extmod/deflate_compress.py: Add a test for optimal compression.
Signed-off-by: Damien George <damien@micropython.org>
2023-11-30 12:13:29 +11:00
Andrew Leech 4cf741062b extmod/vfs_reader: Add file ioctl to set read buffer size.
Can be used to speed up importing a file from a vfs based filesystem.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2023-11-09 11:20:31 +11:00
Jim Mussared 78f4f30cb1 tests/extmod/asyncio_as_uasyncio.py: Fix qstr order dependency.
This test depends on the order in which qstrs are stored in ROM, which
affects the order in which `dir()` will probe the object to see what it
supports.  Because of the lazy-loading in asyncio/__init__.py, if it
tries to do e.g. `wait_for_ms` before `funcs` then it will import funcs,
making `funcs` later succeed. But in the other way around, `funcs` will
initially not be found.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-30 11:10:02 +11:00
Christian Walther 7be16e0540 extmod/vfs_posix: Additional tests for coverage of error cases.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
2023-10-19 16:21:09 +02:00
Christian Walther be28829ae8 extmod/vfs_posix: Fix getcwd() on non-root VFS.
The unwritten API contract expected of a VFS.getcwd() by mp_vfs_getcwd()
is that its return value should be either "" or "/" when the CWD is at
the root of the VFS and otherwise start with a slash and not end with a
slash. This was not correctly implemented in VfsPosix for instances with
a non-empty root - the required leading slash, if any, was cut off
because the root length includes a trailing slash. This would result in
missing slashes in the middle of the return value of os.getcwd() or in
uninitialized garbage from beyond a string's null terminator when the
CWD was at the VFS root.

Signed-off-by: Christian Walther <cwalther@gmx.ch>
2023-10-19 16:21:09 +02:00
Christian Walther 0c4fb16871 extmod/vfs_posix: Fix relative paths on non-root VFS.
The unwritten API contract expected of a VFS by mp_vfs_lookup_path() is
that paths passed in are relative to the root of the VFS if they start
with '/' and relative to the current directory of the VFS otherwise.
This was not correctly implemented in VfsPosix for instances with a
non-empty root - all paths were interpreted relative to the root. Fix
that. Since VfsPosix tracks its CWD using the "external" CWD of the Unix
process, the correct handling for relative paths is to pass them through
unmodified.

Also, when concatenating absolute paths, fix an off-by-one resulting in
a harmless double slash (the root path already has a trailing slash).

Signed-off-by: Christian Walther <cwalther@gmx.ch>
2023-10-19 16:21:09 +02:00
Christian Walther 5f7065f57a extmod/vfs_posix: Fix accidentally passing tests.
These tests test an unrealistic situation and only pass by accident due
to a bug. The upcoming fix for the bug would make them fail.

The unrealistic situation is that VfsPosix methods are called with
relative paths while the current working directory is somewhere outside
of the root of the VFS. In the intended use of VFS objects via
os.mount() (as opposed to calling methods directly as the tests do),
this never happens, as mp_vfs_lookup_path() directs incoming calls to
the VFS that contains the CWD.

Make the testing situation realistic by changing the working directory
to the root of the VFS before calling methods on it, as the subsequent
relative path accesses expect.

Thanks to the preceding commit, the tests still pass, but still for the
wrong reason. The following commit "Fix relative paths on non-root VFS"
will make them pass for the correct reason.

Signed-off-by: Christian Walther <cwalther@gmx.ch>
2023-10-19 16:21:08 +02:00
Christian Walther e3ba6f952b extmod/vfs_posix: Fix relative root path.
A VfsPosix created with a relative root path would get confused when
chdir() was called on it and become unable to properly resolve absolute
paths, because changing directories effectively shifted its root. The
simplest fix for that would be to say "don't do that", but since the
unit tests themselves do it, fix it by making a relative path absolute
before storing it.

Signed-off-by: Christian Walther <cwalther@gmx.ch>
2023-10-19 16:21:08 +02:00
Jim Mussared 8eb7721b4a extmod/modframebuf: Remove FrameBuffer1 from natmod build.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-16 12:18:43 +11:00
Jim Mussared 03a9fa227d extmod/modframebuf: Fix FrameBuffer get-buffer implementation.
This wasn't correctly accounting for the bits-per-pixel and was returning a
bufinfo struct with the incorrect length.  Instead, just forward directly
to the underlying buffer object.

Fixes issue #12563.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-16 12:18:08 +11:00
Jim Mussared d040478d8a extmod/modframebuf: Validate FrameBuffer bounds against input buffer.
This ensures that the buffer is large enough for the specified width,
height, bits-per-pixel, and stride.

Also makes the legacy FrameBuffer1 constructor re-use the FrameBuffer
make_new to save some code size.

Fixes issue #12562.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-16 12:17:51 +11:00
Damien George 05cb1406ad extmod/moductypes: Validate that uctypes.struct addr argument is an int.
Fixes issue #12660.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-12 15:47:15 +11:00
Jim Mussared fae83a6b4d tests/extmod/asyncio_threadsafeflag.py: Update for unix select.
1. Remove the skip for detecting support for polling user-defined objects
   as this is always possible now on all ports.
2. Don't print when the scheduled task runs as the ordering of this
   relative to the other prints is dependent on other factors (e.g. if
   using the native emitter).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-29 17:58:40 +10:00
Damien George 62c3033ba6 tests/extmod/vfs_fat_finaliser.py: Tweak test so files are collected.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-29 12:02:36 +10:00
Damien George a7e2a6d9f2 tests/extmod/ssl_cadata.py: Skip test on axtls.
The axtls bindings don't support this.

Signed-off-by: Damien George <damien@micropython.org>
2023-09-29 12:01:50 +10:00
Damien George a33766880e tests/extmod/deflate_decompress.py: Skip test when not enough memory.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-29 12:01:04 +10:00
Damien George 397697aa83 py/persistentcode: Always close reader even if an exception is raised.
Fixes issue #3874.

Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Damien George 5e122b11ea py/parse: Always free lexer even if an exception is raised.
Fixes issue #3843.

Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Damien George 88518009ce tests/extmod/select_poll_eintr.py: Improve robustness of test.
Increase allowed range of dt_ms, and print it in case of failure.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-15 00:08:51 +10:00
Damien George f6af48416d extmod/modselect: Properly track number of poll objects that are fd's.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-14 18:48:47 +10:00
Damien George d529c20674 extmod/modssl_mbedtls: Fix ioctl of a socket in closed/error state.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-09 13:22:57 +10:00
Damien George 20d3a6b196 extmod/modssl_mbedtls: Reject ioctls that are not supported.
An SSL stream can only handle CLOSE and POLL ioctls.  Other ones do not
make sense, or at least it doesn't make sense to pass the ioctl request
directly down to the underlying stream.

In particular MP_STREAM_GET_FILENO should not be passed to the underlying
stream because the SSL stream is not directly related to a file descriptor,
and the SSL stream must handle the polling itself.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-09 13:22:32 +10:00
Damien George 218242d1de tests/extmod: Skip select/socket tests if they can't create UDP socket.
Some targets (eg PYBV10) have the socket module but are unable to create
UDP sockets without a registered NIC.  So skip UDP tests on these targets.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:39:29 +10:00
Damien George 6b78a1bf00 tests/extmod: Add coverage tests for select module.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:37:48 +10:00
Jim Mussared 8b315ef0d8 tests/extmod: Add deflate.DeflateIO tests.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:32:42 +10:00
Jim Mussared add1200343 all: Remove the zlib module.
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:48:29 +10:00
Damien George 9900eaa269 tests/extmod: Add test for passing cadata into ssl.wrap_socket().
For coverage.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-26 16:34:41 +10:00
Damien George b50da3dbd7 tests/extmod: Add tests for ssl.SSLContext.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-26 16:34:41 +10:00
Damien George e8a4c1dd53 extmod/modssl: Add SSLContext class.
This commit adds the SSLContext class to the ssl module, and retains the
existing ssl.wrap_socket() function to maintain backwards compatibility.

CPython deprecated the ssl.wrap_socket() function since CPython 3.7 and
instead one should use ssl.SSLContext().wrap_socket().  This commit makes
that possible.

For the axtls implementation:
- ssl.SSLContext is added, although it doesn't hold much state because
  axtls requires calling ssl_ctx_new() for each new socket
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added

For the mbedtls implementation:
- ssl.SSLContext is added, and holds most of the mbedtls state
- ssl.verify_mode is added (getter and setter)
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added

The signatures match CPython:
- SSLContext(protocol)
- SSLContext.wrap_socket(sock, *, server_side=False,
    do_handshake_on_connect=True, server_hostname=None)

The existing ssl.wrap_socket() functions retain their existing signature.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-26 16:34:41 +10:00
Duncan Lowther 41c91422f0
tests/extmod/framebuf: Fix buffer size issues.
Tests framebuf1 and framebuf2 do not take the need for byte-aligned
strides into consideration when calculating buffer lengths.
Accordingly, the buffers allocated are slightly too small. Fixed
buffer length calculations.

Signed-off-by: Duncan Lowther <Duncan.Lowther@glasgow.ac.uk>
2023-06-21 09:49:03 +01:00
Duncan Lowther bc2ed8c55a
tests/extmod/uctypes_array_assign_le: Fix buffer.
Structure descriptor in test extmod/uctypes_array_assign_le
is 6 bytes long, due to member "arr3" having length 4
(2 * UINT16) and offset 2, but only 5 bytes are allocated.
Increased buffer length to 6 bytes.

Signed-off-by: Duncan Lowther <Duncan.Lowther@glasgow.ac.uk>
2023-06-21 09:48:37 +01:00
Jim Mussared ca79b49619 extmod/asyncio/uasyncio.py: Add backwards-compatible uasyncio alias.
This allows existing code that does `import uasyncio` or
`import uasyncio as asyncio` to continue working.

It uses the same lazy-loading as asyncio to prevent loading of unused
features.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-19 18:37:34 +10:00
Jim Mussared 6027c41c8f tests: Rename uasyncio to asyncio.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-19 17:33:03 +10:00
Jim Mussared a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared 4216bc7d13 tests: Replace umodule with module everywhere.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Damien George b1229efbd1 all: Fix spelling mistakes based on codespell check.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
Damien George a4672149b6 tests/extmod/vfs_fat_ilistdir_del.py: Use 512-byte erase block size.
Following other vfs_fat tests, so the test works on ports like stm32 that
only support 512-byte block size.

Signed-off-by: Damien George <damien@micropython.org>
2023-03-29 23:15:16 +11:00
Jeremy Rand d677023b3d extmod/vfs_posix: Do not filter '..*' in ilistdir when filtering '..'.
When iterating over os.ilistdir(), the special directories '.' and '..'
are filtered from the results.  But the code inadvertently also filtered
any file/directory which happened to match '..*'.  This change fixes the
filter.

Fixes issue #11032.

Signed-off-by: Jeremy Rand <jeremy@rand-family.com>
2023-03-21 16:13:10 +11:00
Damien George d99ebb310c tests/extmod: Skip vfs tests if target doesn't have enough memory.
Signed-off-by: Damien George <damien@micropython.org>
2023-02-09 15:32:06 +11:00
Damien George b1123a54c1 tests/extmod: Get DecompIO test running on low-memory targets.
By changing the zlib header so that it uses a small (256 byte) window.

Signed-off-by: Damien George <damien@micropython.org>
2023-02-09 15:20:25 +11:00
Jim Mussared 8b27482692 top: Update Python formatting to black "2023 stable style".
See https://black.readthedocs.io/en/stable/the_black_code_style/index.html

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-02-02 12:51:03 +11:00
Damien George fc745d85fe tests/extmod/ure_namedclass: Add tests for named classes in class sets.
Signed-off-by: Damien George <damien@micropython.org>
2023-01-18 09:15:32 +11:00
Damien Tournoud ed58d6e4ce extmod/modussl_mbedtls: Fix support for ioctl(MP_STREAM_POLL).
During the initial handshake or subsequent renegotiation, the protocol
might need to read in order to write (or conversely to write in order
to read). It might be blocked from doing so by the state of the
underlying socket (i.e. there is no data to read, or there is no space
to write).

The library indicates this condition by returning one of the errors
`MBEDTLS_ERR_SSL_WANT_READ` or `MBEDTLS_ERR_SSL_WANT_WRITE`. When that
happens, we need to enforce that the next poll operation only considers
the direction that the library indicated.

In addition, mbedtls does its own read buffering that we need to take
into account while polling, and we need to save the last error between
read()/write() and ioctl().
2022-12-15 12:06:22 -08:00
TPReal 002f54ab4e tests/extmod/framebuf_scroll: Add tests for FrameBuffer scrolling.
Includes a currently-failing test of scrolling by at least the size of the
buffer.
2022-12-09 16:47:08 +11:00
Damien George 965a87b53c tests/extmod: Add test for sleep_ms value that overflows ticks.
Addresses #9516.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-14 16:10:38 +11:00
Damien George 815920c87f extmod/utime_mphal: Make ticks_add check for overflow of delta.
Work done in collaboration with @jimmo.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-14 15:54:53 +11:00
stijn 9ae8d38204 extmod/vfs_posix_file: Implement finaliser for files.
Prevent handle leaks when file objects aren't closed explicitly and
fix some MICROPY_CPYTHON_COMPAT issues: this wasn't properly adhered
to because #ifdef was used so it was always on, and closing files
multiple times should be avoided unconditionally.
2022-09-19 23:44:50 +10:00