RFC: ABI support for special memory area

[H.J. Lu]

On Fri, Mar 3, 2017 at 4:28 AM, Suprateeka R Hegde
<hegdesmailbox@gmail.com> wrote:
> On 02-Mar-2017 09:34 PM, H.J. Lu wrote:
>>
>> On Thu, Mar 2, 2017 at 7:16 AM, Suprateeka R Hegde
>> <hegdesmailbox@gmail.com> wrote:
>>>
>>> On 23-Feb-2017 09:49 PM, H.J. Lu wrote:
>>>>
>>>>
>>>>  The default implementation of __gnu_mbind_setup is
>>>>
>>>> int
>>>> __gnu_mbind_setup (unsigned int type, void *addr, size_t length)
>>>> {
>>>>   return 0;
>>>> }
>>>>
>>>> which can be overridden by a different implementation at link-time.
>>>>
>>>
>>> Since this is a design that allows vendor specific extension and
>>> implementation, would it OK if we make it more generic?
>>
>>
>> Yes.
>>
>>> Instead of a fixed 3 arguments (type, addr, len), how about something
>>> like a
>>> pointer to a generic MBIND_CONTEXT struct (say of type
>>> __gnu_mbind_context
>>> defined)?  And let the implementation define the actual struct.
>>
>>
>> We can add more arguments.  But they must be predefined since
>> __gnu_mbind_setup is called from ld.so which must know what to
>> pass to __gnu_mbind_setup.
>
>
> <snip>
>
>> __gnu_mbind_setup is platform specific.  We can pass as
>> much as we need to __gnu_mbind_setup.  But they have to be fixed.
>
>
> I didnt understand this. Predefined/fixed where and how? As part of the ABI
> support? Or part of the implementation?

The interface is fixed so that it can be called from ld.so.   But its
implementation
is platform specific.

> If it is going to part of ABI, then we need to finalize and define it now.
> If it is part of the implementation, then __gnu_mbind_setup must be
> documented in the ABI as just a guideline to the implementer.
>
> What I meant by being generic is to have flexibility in number of arguments.
> Here is an incomplete/pseudo code of what I am trying to tell:
>
> enum __gnu_mbind_instance {
>    GNU_MBIND_DEFAULT,
>    GNU_MBIND_OVERRIDDEN
> };
>
> typedef struct {
>    __gnu_mbind_instance mb_inst; // mandatory. ABI specified
> #ifdef VENDOR_1
>    type1 identifier1; // optional implementation defined
>    type2 identifier2; // optional implementation defined
>    ...
>    typeN identifierN; // optional implementation defined
> #endif
> // Add vendor/implementation as necessary
> } __gnu_mbind_context;
>
> int __gnu_mbind_setup(__gnu_mbind_context*);
>
> Now, for MCDRAM instance, from my understanding:
>
> typedef struct {
>    __gnu_mbind_instance mb_inst;
> #ifdef VENDOR_MCDARM
>    unsigned int type;
>    void *addr;
>    size_t length;
> #endif
> } __gnu_mbind_context;
>
> int
> __gnu_mbind_setup (__gnu_mbind_context* mbind_context)
> {
>    /* Even if real implementation exist, check and allow disabling special
> memory bindings */
>    if (mbind_context->mb_inst == GNU_MBIND_DEFAULT) {
>       return 0;
>    }
>    else {
>       // return real_implementation_result;
>    }
> }
>
> If that is not what you are telling, I want to understand how to use it
> vendor specific way.

__gnu_mbind_setup is called from ld.so.  Since there is only one ld.so,
it needs to know what to pass to __gnu_mbind_setup.  Not all arguments
have to be used by all implementations nor all memory types.

>
>>> I would like to handle NVM/NVMe (long back I had mentioned about
>>> PT_PERSISTENT) through this MBIND and my implementation of handling
>>> NVM/NVMe
>>> needs more data to be passed to such "setup" functions.
>>
>>
>> I call it MBIND since a MBIND segment is inside a LOAD segment and
>> my real __gnu_mbind_setup will call mbind to move a MBIND region to
>> a NUMA node after it has been loaded and relocated. We can give it
>> a different name if you have a better one.
>
>
> I dont have any comment on this MBIND naming. It sounds good.
>
> --
> Supra



-- 
H.J.