Commit Graph

229 Commits

Author SHA1 Message Date
Dave Hylands 9f76dcd682 py: Prevent many extra vstr allocations.
I checked the entire codebase, and every place that vstr_init_len
was called, there was a call to mp_obj_new_str_from_vstr after it.

mp_obj_new_str_from_vstr always tries to reallocate a new buffer
1 byte larger than the original to store the terminating null
character.

In many cases, if we allocated the initial buffer to be 1 byte
longer, we can prevent this extra allocation, and just reuse
the originally allocated buffer.

Asking to read 256 bytes and only getting 100 will still cause
the extra allocation, but if you ask to read 256 and get 256
then the extra allocation will be optimized away.

Yes - the reallocation is optimized in the heap to try and reuse
the buffer if it can, but it takes quite a few cycles to figure
this out.

Note by Damien: vstr_init_len should now be considered as a
string-init convenience function and used only when creating
null-terminated objects.
2015-07-06 17:29:27 +01:00
Paul Sokolovsky f44cc517a2 objstr: Add note that replace() is nicely optimized.
Doesn't allocate memory and returns original string if no replacements are
to be made.
2015-06-26 17:35:12 +03:00
Damien George 79474c6b16 py: Remove unnecessary extra handling of padding of nan/inf.
C's printf will pad nan/inf differently to CPython.  Our implementation
originally conformed to C, now it conforms to CPython's way.

Tests for this are also added in this patch.
2015-05-28 14:22:12 +00:00
Damien George 44e7cbf019 py: Clean up declarations of str type/funcs that are also in unicode.
Background: trying to make an amalgamation of all the code gave some
errors with redefined types and inconsistent use of static.
2015-05-17 16:44:24 +01:00
Damien George c2a4e4effc py: Convert hash API to use MP_UNARY_OP_HASH instead of ad-hoc function.
Hashing is now done using mp_unary_op function with MP_UNARY_OP_HASH as
the operator argument.  Hashing for int, str and bytes still go via
fast-path in mp_unary_op since they are the most common objects which
need to be hashed.

This lead to quite a bit of code cleanup, and should be more efficient
if anything.  It saves 176 bytes code space on Thumb2, and 360 bytes on
x86.

The only loss is that the error message "unhashable type" is now the
more generic "unsupported type for __hash__".
2015-05-12 22:46:02 +01:00
Damien George ede0f3ab3d py: Add optional code to check bytes constructor values are in range.
Compiled in only if MICROPY_CPYTHON_COMPAT is set.

Addresses issue #1093.
2015-04-23 15:28:18 +01:00
Damien George 7f9d1d6ab9 py: Overhaul and simplify printf/pfenv mechanism.
Previous to this patch the printing mechanism was a bit of a tangled
mess.  This patch attempts to consolidate printing into one interface.

All (non-debug) printing now uses the mp_print* family of functions,
mainly mp_printf.  All these functions take an mp_print_t structure as
their first argument, and this structure defines the printing backend
through the "print_strn" function of said structure.

Printing from the uPy core can reach the platform-defined print code via
two paths: either through mp_sys_stdout_obj (defined pert port) in
conjunction with mp_stream_write; or through the mp_plat_print structure
which uses the MP_PLAT_PRINT_STRN macro to define how string are printed
on the platform.  The former is only used when MICROPY_PY_IO is defined.

With this new scheme printing is generally more efficient (less layers
to go through, less arguments to pass), and, given an mp_print_t*
structure, one can call mp_print_str for efficiency instead of
mp_printf("%s", ...).  Code size is also reduced by around 200 bytes on
Thumb2 archs.
2015-04-16 14:30:16 +00:00
Paul Sokolovsky 8b7faa31e1 objstr: split(None): Fix whitespace properly. 2015-04-12 00:17:57 +03:00
Damien George 2801e6fad8 py: Some trivial cosmetic changes, for code style consistency. 2015-04-04 15:53:11 +01:00
Paul Sokolovsky 7f59b4b2ca objstr: Fix bugs introduced by inability to have shadow variables.
Warnings lead to programming errors - as expected.
2015-04-04 01:55:40 +03:00
Paul Sokolovsky acf6aec71c objstr: Avoid variable shadowing. 2015-04-04 01:24:59 +03:00
Paul Sokolovsky ac2f7a7f6a objstr: Add .splitlines() method.
splitlines() occurs ~179 times in CPython3 standard library, so was
deemed worthy to implement. The method has subtle semantic differences
from just .split("\n"). It is also defined as working for any end-of-line
combination, but this is currently not implemented - it works only with
LF line-endings (which should be OK for text strings on any platforms,
but not OK for bytes).
2015-04-04 00:09:48 +03:00
Paul Sokolovsky 8705171233 objstr: Expose mp_obj_str_split() for reuse in other modules. 2015-03-23 22:43:37 +02:00
Damien George fa1edff006 py: Remove unnecessary and unused sgn argument from pfenv_print_mp_int. 2015-03-14 22:32:40 +00:00
Paul Sokolovsky 194117a066 objstr: Fix bytes creation from array of long ints. 2015-02-09 12:11:49 +08:00
Damien George 827b0f747b py: Change vstr_null_terminate -> vstr_null_terminated_str, returns str. 2015-01-29 13:57:23 +00:00
Damien George 0d3cb6726d py: Change vstr so that it doesn't null terminate buffer by default.
This cleans up vstr so that it's a pure "variable buffer", and the user
can decide whether they need to add a terminating null byte.  In most
places where vstr is used, the vstr did not need to be null terminated
and so this patch saves code size, a tiny bit of RAM, and makes vstr
usage more efficient.  When null termination is needed it must be
done explicitly using vstr_null_terminate.
2015-01-28 23:43:01 +00:00
Paul Sokolovsky bbd9251bac py: bytes(): Make sure we add values as bytes, not as chars. 2015-01-28 22:29:07 +02:00
Damien George 98e3a64694 py: Remove duplicated mp_obj_str_make_new function from objstrunicode.c. 2015-01-28 14:14:57 +00:00
Paul Sokolovsky 344e15b1ae objstr: Remove code duplication and unbreak Windows build.
There was really weird warning (promoted to error) when building Windows
port. Exact cause is still unknown, but it uncovered another issue:
8-bit and unicode str_make_new implementations should be mutually exclusive,
and not built at the same time. What we had is that bytes_decode() pulled
8-bit str_make_new() even for unicode build.
2015-01-23 02:15:56 +02:00
Paul Sokolovsky 6113eb2f33 objstr*: Use separate names for locals_dict of 8-bit and unicode str's.
To somewhat unbreak -DSTATIC="" compile.
2015-01-23 02:05:58 +02:00
Damien George 77089bebd4 py: Add comments for vstr_init and mp_obj_new_str. 2015-01-21 23:18:02 +00:00
Damien George 05005f679e py: Remove mp_obj_str_builder and use vstr instead.
With this patch str/bytes construction is streamlined.  Always use a
vstr to build a str/bytes object.  If the size is known beforehand then
use vstr_init_len to allocate only required memory.  Otherwise use
vstr_init and the vstr will grow as needed.  Then use
mp_obj_new_str_from_vstr to create a str/bytes object using the vstr
memory.

Saves code ROM: 68 bytes on stmhal, 108 bytes on bare-arm, and 336 bytes
on unix x64.
2015-01-21 23:18:02 +00:00
Damien George 0b9ee86133 py: Add mp_obj_new_str_from_vstr, and use it where relevant.
This patch allows to reuse vstr memory when creating str/bytes object.
This improves memory usage.

Also saves code ROM: 128 bytes on stmhal, 92 bytes on bare-arm, and 88
bytes on unix x64.
2015-01-21 23:17:27 +00:00
Damien George ff8dd3f486 py, unix: Allow to compile with -Wunused-parameter.
See issue #699.
2015-01-20 12:47:20 +00:00
Damien George 50912e7f5d py, unix, stmhal: Allow to compile with -Wshadow.
See issue #699.
2015-01-20 11:55:10 +00:00
Damien George 963a5a3e82 py, unix: Allow to compile with -Wsign-compare.
See issue #699.
2015-01-16 17:47:07 +00:00
Damien George 0178aa9a11 py, unix: Allow to compile with -Wdouble-promotion.
Ref issue #699.
2015-01-12 21:56:35 +00:00
Damien George e233a55a29 py: Remove unnecessary BINARY_OP_EQUAL code that just checks pointers.
Previous patch c38dc3ccc7 allowed any
object to be compared with any other, using pointer comparison for a
fallback.  As such, existing code which checked for this case is no
longer needed.
2015-01-11 21:07:15 +00:00
Paul Sokolovsky ff8e35b42e objstr: Common subexpression elimination for vstr_str(field_name). 2015-01-04 13:23:44 +02:00
Paul Sokolovsky c114496641 objstr: Implement kwargs support for str.format(). 2015-01-04 00:26:31 +02:00
Damien George 51dfcb4bb7 py: Move to guarded includes, everywhere in py/ core.
Addresses issue #1022.
2015-01-01 20:32:09 +00:00
Paul Sokolovsky 2c75665445 objstr: Fix %d-formatting of floats. 2014-12-31 02:21:19 +02:00
Damien George c55a4d82cf py: Make bytes objs work with more str methods; add tests. 2014-12-24 20:28:30 +00:00
Damien George 81836c28b3 py: Use str_to_int function in more places to reduce code size. 2014-12-21 21:07:03 +00:00
Damien George b4fe6e28eb py: Fix function type: () -> (void). 2014-12-10 18:05:42 +00:00
Damien George 32ef3a3517 py: Allow bytes/bytearray/array to be init'd by buffer protocol objects.
Behaviour of array initialisation is subtly different for bytes,
bytearray and array.array when argument has buffer protocol.  This patch
gets us CPython conformant (except we allow initialisation of
array.array by buffer with length not a multiple of typecode).
2014-12-04 15:46:14 +00:00
Damien George 6f5eb84c19 py: #if guard str_make_new when not needed. 2014-11-27 16:55:47 +00:00
Damien George 1e9a92f84f py: Use shorter, static error msgs when ERROR_REPORTING_TERSE enabled.
Going from MICROPY_ERROR_REPORTING_NORMAL to
MICROPY_ERROR_REPORTING_TERSE now saves 2020 bytes ROM for ARM Thumb2,
and 2200 bytes ROM for 32-bit x86.

This is about a 2.5% code size reduction for bare-arm.
2014-11-06 17:36:16 +00:00
Damien George be8e99c7d4 py: Allow bytes object as argument to some str methods.
This turns failing assertions to type exceptions for things like
b"123".find(...).  We still don't support operations like this on bytes
objects (unlike CPython), but at least it no longer crashes.
2014-11-05 16:45:54 +00:00
Damien George a65c03c6c0 py: Allow +, in, and compare ops between bytes and bytearray/array.
Eg b"123" + bytearray(2) now works.  This patch actually decreases code
size while adding functionality: 32-bit unix down by 128 bytes, stmhal
down by 84 bytes.
2014-11-05 16:30:34 +00:00
Paul Sokolovsky e62a0fe367 objstr: Allow to convert any buffer proto object to str.
Original motivation is to support converting bytearrays, but easier to just
support buffer protocol at all.
2014-10-31 00:03:53 +02:00
Paul Sokolovsky 31619cc589 py: mp_obj_str_get_str(): Work with bytes too.
It should be fair to say that almost in all cases where some API call
expects string, it should be also possible to pass byte string. For example,
it should be open/delete/rename file with name as bytestring. Note that
similar change was done quite a long ago to mp_obj_str_get_data().
2014-10-31 00:00:39 +02:00
Damien George 3aa09f5784 py: Use MP_OBJ_NULL instead of NULL in a few places. 2014-10-23 12:06:53 +01:00
Damien George 20f59e182e py: Make mp_const_empty_bytes globally available. 2014-10-21 21:02:56 +01:00
Damien George 39dc145478 py: Change [u]int to mp_[u]int_t in qstr.[ch], and some other places.
This should pretty much resolve issue #50.
2014-10-03 19:52:22 +01:00
Damien George cde0ca21bf py: Simplify JSON str printing (while still conforming to JSON spec).
The JSON specs are relatively flexible and allow us to use one function
to print strings, be they ascii, bytes or utf-8 encoded.
2014-09-25 17:35:56 +01:00
Damien George 612045f53f py: Add native json printing using existing print framework.
Also add start of ujson module with dumps implemented.  Enabled in unix
and stmhal ports.  Test passes on both.
2014-09-17 22:56:34 +01:00
Damien George 4abff7500f py: Change uint to mp_uint_t in runtime.h, stackctrl.h, binary.h.
Part of code cleanup, working towards resolving issue #50.
2014-08-30 14:59:21 +01:00
Damien George 4d91723587 py: Remove use of int type in obj.h.
Part of code cleanup, working towards resolving issue #50.
2014-08-30 14:28:06 +01:00