what is the logic of populate?

[ng@piments.com]

Hi,

first of all, thanks for the reply. It seems much of the problem was me 
tripping myself up having started to build a clean source dir and then 
confusing the two.

Now I'll reply to a couple of points where I think ct-ng could be made 
easier to understand and note things that have caused me some confusion.



Yann E. MORIN wrote:
> Hello!
> 
> On Sunday 31 May 2009 15:27:10 ng@piments.com wrote:
>> arm-unknown-linux-gnueabi-populate -v -f -s root -d root-image
> 
> Why do you need to use '-f' at all?

Basically because it fails to create destdir if I don't. This way I 
don't need to check and conditionally do a mkdir myself.

Is there a good reason why this is not the default behaviour?

When I first ran populate is returned silently. I took this to be a 
successful execution , in fact it had done nothing.

You explain that this is one of the sanity checks. But if it fails a 
sanity check and aborts, it would seem best to indicate that it 
terminated without doing what was requested and to state what the 
problem was.

It seems curious that it will happily rm -rf destdir without warning but 
  drops out if it needs to do a mkdir. If anything, I would expect the 
opposite: -f being needed to authorise scrapping an existing structure.

At least the mkdir would seem safe to do without the -f switch.

> 
>> it created a very sleek and light root-image/lib but failed to find 
>> libwrap and hence failed to copy it.
>> #ls /back/ts/root2/usr/lib/libwrap*
> 
> Your command line is using "root" as the source dir, but you have
> libwrap in "root2". Maybe that's the reason...
> 
>> In looking further I found it had copied /var/spool/cron/crotab but NOT 
>> that actual crontab files therein. So I have not cron tasks defined.
> 
> Same root vs. root2 mismatch?
> 
>> bash-4.0#ls -l /back/ts/root2/sbin/init
>> -rwxr-xr-x 1 root root 31211 2009-05-22 22:39 /back/ts/root2/sbin/init
>> bash-4.0#ls -l /back/ts/root-image/sbin/init
>> lrwxrwxrwx 1 root root 14 2009-05-31 13:38 /back/ts/root-image/sbin/init 
>> -> ../bin/busybox
>> So it has replaced the REAL sysvinit with a symlink to busybox !? No 
>> wonder it won't boot. What is going on here?
> 
> populate is not supposed to do that, indeed.
> root vs. root2 mismatch, again?
> 
>> In order for populate to determine the required libs , it must have all 
>> programs (such as init) in place in the source tree. If it is then going 
>> to ignore the installed progs and arbitrarily create symlinks to other 
>> executables it's seems difficult to have confidence in the result.
> 
> populate does not do that. Look at the code, it's quite straightforward.
> 
> 1) some sanity checks:
>    - dest and source are specified
>    - source exists
>    - unless forced, dest does not exist
>    - source != dest
> 
> 2) copy source to dest
> 
> 3) add forced libraries to dest
>    - build the list from -l and -L options
>    - get forced libraries from sysroot (see below for heuristics)
> 
> 4) add all missing libraries
>    - scan dest for every ELF files that are 'executables' or 'shared object'
>    - list the "NEEDED Shared library" fields
>      - check if the library is already in dest/lib or dest/usr/lib
>      - if not, get the library from sysroot
>        - if it's in sysroot/lib, copy it to dest/lib
>        - if it's in sysroot/usr/lib, copy it to dest/usr/lib
>        - in both cases, use the SONAME of the library to create the file
>          in dest
>        - if it was not found in the sysroot, this is an error.
> 
> That's all. No symlinking, no file-name mangling (besides using the SONAME), 
> no file removal...
> 
> Maybe we could add some other paths to check for, eg. usr/local/lib...
> 

Yes this may be a good idea.

>> Apparently I do not understand the exact function of populate. This may 
>> be due in part to the very brief usage help that appears to be the only 
>> documentation.
> 
> What do you think of the above? If it makes things clear, then I'll
> add that to the doc.
> 
> I'm using it, and it does work. The only gotcha you may encounter is if
> you are using a non-english locale, as I forgot to force LC_ALL and LANG
> before I run readelf, and that may imply missing the NEEDED fileds...
> Will fix that.
> 
>> Is this information available anywhere?
> 
> I though that docs/overview.txt and the help text were enough. Moreover,
> the code is quite simple, and (hopefuly) well commented... Apparently,
> that was not enough...

overview.txt is a good overview but I found it lacked detail I needed to 
know. Adding something like the four point summary you gave above would 
be very helpful.

I'm not a fan of doc==src . If I find I have to start ploughing through 
source code to find out how a tool works I'd generally look for another 
tool or do the job by hand.



One final point about overview.txt, I found the description of the 
arguments confusing. Now I know what it does it makes sense. When I was 
reading this find out what it did it was confusing.

 >>
When your root directory is ready, it is still missing some important 
bits: the toolchain's libraries. To populate your root directory with 
those libs, just run:
   your-target-tuple-populate -s /your/root -d /your/root-populated
 >>

Confusion: "to populate your root" does not mean populate /your/root

In fact you never populate "your root" directory as it is called here, 
and it is never used as an image of the root as you describe it. The 
following changes to that text may make it easier to understand for 
those who do not already know how populate works:

c?/your/root?/your/root-tmp?g

c/To populate your root directory with/To create a copy of your 
temporary root directory and populate it with/


I hope these comments are useful. Thanks once again for your help.

regards, Peter.



> 
> Regards,
> Yann E. MORIN.
> 


--
For unsubscribe information see http://sourceware.org/lists.html#faq