[PATCH] libctf: ctf_member_next needs to return (ssize_t)-1 on error

[Torbjörn SVENSSON]

The function ctf_member_next should return (ssize_t)-1 on
error. As the function ctf_set_errno returns (ctf_id_t)-1L and that is
then casted to "unsigned long" as it's the return type of the function,
it's not compatible and causes the value 0xffffffff to be returned on
64-bit Windows builds. As a result, the check for a negative value in
ctf_dedup_rhash_type will never be true and a resulting infinit loop is
created.

This was found testing an arm-none-eabi toolchain built with
x86_64-w64-mingw32. If the same source tree is built with
i686-w64-mingw32, everything appears to be working correctly.

Signed-off-by: Torbjörn SVENSSON <torbjorn.svensson@foss.st.com>
Co-Authored-By: Yvan ROUX <yvan.roux@foss.st.com>
---
 libctf/ctf-types.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index c20ff825d9a..058b647ba9a 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -233,7 +233,8 @@ ctf_member_next (ctf_dict_t *fp, ctf_id_t type, ctf_next_t **it,
  end_iter:
   ctf_next_destroy (i);
   *it = NULL;
-  return ctf_set_errno (ofp, ECTF_NEXT_END);
+  ctf_set_errno (ofp, ECTF_NEXT_END);
+  return -1;
 }
 
 /* Iterate over the members of an ENUM.  We pass the string name and associated
-- 
2.25.1