[PATCH v5] libctf: Sanitize error types for PR 30836

[Nick Alcock]

On 5 Oct 2023, Torbjörn SVENSSON verbalised:

> v1 -> v2:
> Changed all functions with signed interger return type to return -1 based on
> comment from Alan.

(integer)

> v2 -> v3:
> Added ctf_set_errno_signed function to return a signed -1 value based on
> comment from Nick.
>
> v3 -> v4:
> - Moved ctf_set_errno_signed function to ctf-inlines.h, renamed it to
> ctf_set_int_errno and converted it to an inline function.
> - Moved ctf_set_errno function to ctf-inlines.h, renamed it to
> ctf_set_type_errno, changed return type to ctf_id_t and converted it to an
> inline function.
> - Updated the changelog entry in the commit message. Is this list really
> required? I don't think it give much information in this patch and 'git log'
> have mixed commits (some have the entry while others don't).
>
> v4 -> v5:
> Updated in accordance with comments from Nick.
> - Changed return type to void for ctf_set_errno.
> - Inline the return on every call to ctf_set_errno.
> - Merged ctf_set_int_errno and ctf_set_type_errno into ctf_set_errno.
> - Droped log entry as it's too many places with this change to make it
>   readable.
> - Corrected a few places where -1 was returned where it should have been
>   CTF_ERR.
>
>
> @Nick: Can you please try this is in your test bench and let me know if it's
> ok for trunk?

... I don't understand why we even have a ctf_set_errno any more. It
does nothing that fp->ctf_errno = FOO doesn't. (Is it just to keep the
commit size down? If so, that seems sensible: either of us can follow it
up with a patch removing ctf_set_errno entirely as redundant.)


Alternatively -- and I promise I'll stop constantly changing my mind in
a moment! it's just this patch seems likely to be pervasive enough that
I'd like to get it right first time -- since nearly all functions in
ctf-api.h return either ctf_id_t or int, we could just do what you did
in the *previous* patch and have two functions for those:

ctf_id_t ctf_set_typed_errno()

for functions returning ctf_id_t, and

int ctf_set_errno()

for functions returning int. The original problem, those few functions
that return other types, would still need to do things "by hand",
probably by setting fp->ctf_errno before returning.

I *think* that would work for all callers and let people just compare
with CTF_ERR for ctf_id_t and unsigned types and with < 0 for all the
others, like they're suposed to be.

That way about half the call sites could stay unchanged, most of the
rest only need one a _typed_ added, and hopefully it's easier to grasp?
plus there is less need to introduce new { ... } just for the sake of
setting the errno value.

We still have a potential problem with functions returning unsigned
types that might be wider than ctf_id_t, since CTF_ERR isn't suitable to
compare against to detect errors for those; thankfully the only one of
those is ctf_archive_count, which cannot return errors :)

Again, really sorry I keep chopping and changing like this, but every
time you do something I think of a possible better way. (Unless I've
missed something, which I probably have.)