Memory consumption of iconv

[Ángel González]

On 21/03/14 16:57, Dirk Bächle wrote:
> Hi Oliver,
>
> On 21.03.2014 12:53, Oliver Becker wrote:
>> Hi,
>>   I am developing a server software which spawns up to thousands of 
>> client
>> processes which use iconv_open (through Qt) to later transform 
>> strings between
>> different encodings. This works like a charm of course but after a 
>> while I
>> recognized that every process used about 1MB (non shared) memory for 
>> the iconv
>> data.
>> This allone is not that much but If you multiply it by the number of 
>> child
>> processes, I get a huge size of multiple GB. If you take into account 
>> that I
>> use the same locales in every child process this is not easy to 
>> understand.
>
> I've opened a related thread for the build system SCons, see
>
>   https://www.sourceware.org/ml/libc-help/2014-01/msg00038.html
>
> . Seems that we both suffer from the same behaviour (all memory of the 
> parent process gets copied for a spawned child), so I'm very 
> interested in any answers you get. ;)
I don't think it's the same case. You were allocating the memory in the 
parent (and changing it).

While Oliver is loading the same file from several processes (which 
_could_ have been shared).

Oliver, maybe you could iconv_open() once in the server? That will 
probably help to get the memory shared. Although its feasibility will 
depend a lont in what you are exactly doing.