Re: Memory (mis)management when writing C extensions?

Guido.van.Rossum@cwi.nl
Mon, 30 Jan 1995 13:38:19 +0100

> -- a NEWOBJ sets the reference count to 1,
> -- most container objects (mapping type, list), perform
> an INCREF() on the inserted object at insertion time,
> (there is the documented exception for set* methods)

Correctly summarized.

> I've looked at the code for dictinsert() and addlistitem() and this
> appears to be the case. However, if I search the python/Modules
> directory for occurances of dictinsert and addlistitem calls, I find
> many cases that appear to be generating unreclaimable objects. For
> example:
>
> timemodule.c: dictinsert(d, "timezone", newintobject((long)timezone));
> timemodule.c: dictinsert(d, "altzone", newintobject((long)altzone));
> timemodule.c: dictinsert(d, "altzone", newintobject((long)timezone-3600));
> timemodule.c: dictinsert(d, "daylight", newintobject((long)daylight));

This is indeed formally a leak, and I've fixed it in 1.2. It's not
really a problem in practice, however, since (in a normal program)
exactly one copy of the time module's dictionary will be created and
it will live until the end of time (:-).

> dbmmodule.c:
> static object *
> dbm_keys(dp, args)
> register dbmobject *dp;
> object *args;
> {
> ...
> for (key = dbm_firstkey(dp->di_dbm); key.dptr;
> key = dbm_nextkey(dp->di_dbm) ) {
> item = newsizedstringobject(key.dptr, key.dsize);
> if ( item == 0 )
> return NULL;
> --->> addlistitem(v, item);
> }
> return v;
> }
>
> In the timemodule.c examples, aren't all the inserted objects memory
> leaks?
>
> In the last example, shouldn't the addlistitem(v,item) be followed by
> a DECREF(item), otherwise its a leak?

Oops, yes, this is a big leak... Thanks for reporting it! Will be
fixed in 1.2.

> Is there a consistent policy about what happens to an item that is
> inserted but the insertion operation fails? That is, is the
> container operation considered atomic with regard to the refcnt
> modification? I recall reading something about it somewhere but
> cannot find it again (my head is swimming!).

In general, yes. Those operations that do their own INCREF when they
insert an object in some data structure won't do the INCREF when
there's a failure. Those operations that expect the caller to have
done the INCREF (or a NEWREF) will explicitly do a DECREF when they
fail. (This only applies to a very small number of functions -- I
think only setlistitem() and settupleitem() follow this pattern.)

--Guido van Rossum, CWI, Amsterdam <mailto:Guido.van.Rossum@cwi.nl>
<http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>