#pypy IRC log for Wednesday, 2010-06-02

antocuni (~antocuni@host143-125-dynamic.25-79-r.retail.telecomitalia.it) joined #pypy.00:18
antocuni (~antocuni@host143-125-dynamic.25-79-r.retail.telecomitalia.it) left irc: Remote host closed the connection00:18
alestan (~perkins@208.81.152.74) left irc: Ping timeout: 264 seconds00:22
lucian (~lucian@cpc3-benw9-2-0-cust344.gate.cable.virginmedia.com) left irc: Quit: Leaving...00:23
alestan (~perkins@208.81.152.74) joined #pypy.00:26
alestan (perkins@208.81.152.74) left #pypy.00:32
nopper (~nopper@host31-13-dynamic.7-87-r.retail.telecomitalia.it) left irc: Quit: quit00:48
ambroff (~ambroff@65-113-99-90.dia.static.qwest.net) left irc: Quit: Ex-Chat01:08
ErKa (~keryell@static-64-22-43-58.albyny.csvoip.net) left irc: Read error: Operation timed out02:26
bitmonk (~jryan@adsl-75-7-0-54.dsl.pltn13.sbcglobal.net) left irc: Quit: bitmonk02:36
qbproger (~qbproger@64.85.204.68.cfl.res.rr.com) joined #pypy.02:54
Nick change: magcius_ -> magcius03:05
qbproger (~qbproger@64.85.204.68.cfl.res.rr.com) left irc: Remote host closed the connection03:13
bigdog (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) joined #pypy.03:19
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) joined #pypy.03:27
wuhy (~chatzilla@121.28.73.42) joined #pypy.03:38
gutworth (~benjamin@64-131-1-181.usfamily.net) left irc: Ping timeout: 265 seconds03:43
bigdog1 (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) joined #pypy.03:49
bigdog (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) left irc: Ping timeout: 272 seconds03:50
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) left irc: Ping timeout: 260 seconds04:05
kursor (~kursor@p57BB10B6.dip0.t-ipconnect.de) joined #pypy.04:19
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) joined #pypy.04:27
panni_ (hannes@ip-95-222-52-93.unitymediagroup.de) left irc: Read error: Connection reset by peer04:29
kursor (~kursor@p57BB10B6.dip0.t-ipconnect.de) left irc: Quit: kursor04:35
SandGorgon (~OmNomNomO@122.162.129.207) joined #pypy.04:37
ly- (~ly-@unaffiliated/ly-) left irc: Quit: ly-04:46
fijalAdeman: ping04:55
Ademanpong04:55
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) left irc: Ping timeout: 240 seconds05:26
bigdog1 (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) left irc: Ping timeout: 240 seconds06:12
DasIch_ (~DasIch@p5DC5EE3E.dip.t-dialin.net) joined #pypy.06:53
DasIch (~DasIch@p5DC5ED07.dip.t-dialin.net) left irc: Ping timeout: 276 seconds06:56
mvt (~mvt@87.213.45.85) joined #pypy.07:04
pemryan (~pem@2001:cc0:201e:107:221:86ff:fe1a:e5aa) joined #pypy.07:18
verte (~9d80dace@gateway/web/freenode/x-dfkgydclkpszofrn) joined #pypy.07:20
ousado (~ousado@p5B3C2A1D.dip0.t-ipconnect.de) left irc: Read error: Connection reset by peer07:26
ousado (~ousado@p5B3C2A1D.dip0.t-ipconnect.de) joined #pypy.07:26
pemryan (~pem@2001:cc0:201e:107:221:86ff:fe1a:e5aa) left irc: Remote host closed the connection07:29
pemryan (~pem@2001:cc0:201e:107:221:86ff:fe1a:e5aa) joined #pypy.07:29
tav (~tav@92.29.54.248) got netsplit.07:33
jcp (~jw@bzflag/contributor/javawizard2539) got netsplit.07:33
stargaming (stargaming@unaffiliated/stargaming) got netsplit.07:33
tav (~tav@92.29.54.248) returned to #pypy.07:39
jcp (~jw@bzflag/contributor/javawizard2539) returned to #pypy.07:39
stargaming (stargaming@unaffiliated/stargaming) returned to #pypy.07:39
wuhy (~chatzilla@121.28.73.42) left irc: Quit: ChatZilla 0.9.86 [Firefox 3.6.3/20100401080539]07:45
dju (dju@fsf/member/dju) joined #pypy.07:50
dju (dju@fsf/member/dju) left irc: Max SendQ exceeded07:51
dju (dju@fsf/member/dju) joined #pypy.07:52
dju (dju@fsf/member/dju) left irc: Quit: Quitte07:58
amaury_ (~5bc3480e@gateway/web/freenode/x-uurjjagxlafxaxtz) joined #pypy.08:07
cwillu_ (~cwillu@cwillu.com) joined #pypy.08:08
e-max__ (~e-max@212.44.158.20) left irc: Ping timeout: 245 seconds08:08
e-max_ (~e-max@217.14.244.16) joined #pypy.08:21
Unhelpful (~quassel@rockbox/developer/Unhelpful) left irc: Read error: Connection reset by peer08:21
Unhelpful (~quassel@rockbox/developer/Unhelpful) joined #pypy.08:22
pemryan (~pem@2001:cc0:201e:107:221:86ff:fe1a:e5aa) left irc: Remote host closed the connection08:26
verte (~9d80dace@gateway/web/freenode/x-dfkgydclkpszofrn) left irc: 08:40
Nick change: cwillu_ -> cwillu08:42
pedronis (~pedronis@46-180.61-188.cust.bluewin.ch) joined #pypy.09:08
pemryan (~pem@2001:cc0:201e:107:221:86ff:fe1a:e5aa) joined #pypy.09:14
pemryan (pem@2001:cc0:201e:107:221:86ff:fe1a:e5aa) left #pypy ("Konversation terminated!").10:16
ErKa (~keryell@static-64-22-43-58.albyny.csvoip.net) joined #pypy.10:26
Hory (~hory@78.96.96.85) joined #pypy.10:45
Hong_MinHee (~dahlia@211.239.163.54) left irc: Ping timeout: 260 seconds10:55
Hong_MinHee (~dahlia@211.239.163.54) joined #pypy.10:55
ErKa (~keryell@static-64-22-43-58.albyny.csvoip.net) left irc: Ping timeout: 248 seconds11:43
fubd (~a@digsby05.rit.edu) left irc: Ping timeout: 276 seconds11:45
fubd (~a@digsby05.rit.edu) joined #pypy.11:48
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) joined #pypy.12:01
arigato (~arigo@c-354be353.022-54-67626719.cust.bredbandsbolaget.se) joined #pypy.12:01
pedronis (~pedronis@46-180.61-188.cust.bluewin.ch) left irc: Ping timeout: 264 seconds12:07
ErKa (~keryell@static-64-22-43-58.albyny.csvoip.net) joined #pypy.12:10
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) left irc: Ping timeout: 252 seconds12:13
amaury_arigato: I found an important thing about our issue with llhelper() last night12:16
amaury_there is a sort of hack to support llhelper on const functions12:17
amaury_but it does not work when several functions are declared with the same name12:17
amaury_only the last can be generated12:17
ErKa (~keryell@static-64-22-43-58.albyny.csvoip.net) left irc: Ping timeout: 276 seconds12:18
arigatoamaury_: not sure I follow12:28
arigatoas far as I can tell, the name of the function has no effect12:32
ErKa (keryell@ted.dyn.cs.rpi.edu) joined #pypy.12:32
amaury_the hack is that on the function passed to llhelper12:33
amaury_we set a "c_name" attribute12:33
arigatowhere?12:34
amaury_in cpyext/api.py12:34
amaury_func.get_wrapper(space).c_name = name12:34
kursor (~kursor@p57BB236B.dip0.t-ipconnect.de) joined #pypy.12:34
amaury_but we use a global dict, keyed by function name12:34
pedronis (~pedronis@46-180.61-188.cust.bluewin.ch) joined #pypy.12:34
amaury_in our case, all the tp_setattro had the same name12:35
arigatoah, but that's an issue "only" local to api.py12:35
amaury_yes12:35
arigatoyou can rename these lambda functions if you need to12:35
arigatothere is even pypy.tool.sourcetools.func_with_new_name() to help12:35
amaury_I know this12:35
amaury_it just took time to understand why my tests passed, but not the translation12:36
arigatoah :-)12:36
magciusso how exactly does the CPyExt work?13:19
magciusDoes it actually parse C and try and reimplement the Python calls?13:20
amaury_it does not parse C13:20
amaury_you have compilers for this13:20
amaury_CPyExt implements the python C functions13:20
amaury_PyString_FromString() &co13:21
amaury_and yes, we implement them by hand13:22
ly- (~ly-@unaffiliated/ly-) joined #pypy.13:31
Alex_Gaynor (~Alex_Gayn@76-206-247-179.lightspeed.cicril.sbcglobal.net) left irc: Quit: Alex_Gaynor13:32
Alex_Gaynor (~alex@76-206-247-179.lightspeed.cicril.sbcglobal.net) joined #pypy.13:34
Hory (~hory@78.96.96.85) left irc: Read error: Connection reset by peer13:35
romao (~Caio@unaffiliated/romao) joined #pypy.13:41
Arfrever (~Arfrever@gentoo/developer/Arfrever) joined #pypy.13:44
copelco (~copelco@h-66-166-135-210.atlngahp.static.covad.net) joined #pypy.13:45
Hory (~hory@78.96.96.85) joined #pypy.13:52
panni_ (hannes@ip-95-222-52-93.unitymediagroup.de) joined #pypy.13:54
ehm (nope@chello089173023218.chello.sk) joined #pypy.14:19
Alex_Gaynorarigato: is test_random an expected fail on blackhole?14:22
ehmo (nope@chello089173023218.chello.sk) left irc: Ping timeout: 240 seconds14:22
arigatono14:23
Alex_Gaynorhttp://paste.pocoo.org/show/221045/ is the tb14:23
tavfellas, please to mod up http://news.ycombinator.com/item?id=1398052 =)14:44
elmom_ (~elmom@vallila-gw.hupnet.helsinki.fi) joined #pypy.14:58
ronny_tav: sup, didnt you have an authormap somewhere?15:16
Shanita (~John@g227200134.adsl.alicedsl.de) left irc: Read error: Connection reset by peer15:18
Shanita (~John@g227200134.adsl.alicedsl.de) joined #pypy.15:19
tavronny_: hey, here's a copy: http://gist.github.com/419236.txt15:19
ErKa (keryell@ted.dyn.cs.rpi.edu) left irc: Ping timeout: 260 seconds15:20
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) joined #pypy.15:21
ronny_arigato: any toughts on setting up a subdir in pypy for puting some converting metadata there?15:23
ronny_at least authormap, maybe some rules15:23
ronny_hmm, who is actually the 'boss'15:24
ronny_wrt repo management and other stuff15:24
kenaan03arigo 10r7502014 07M(pypy/branch/blackhole-improvement/pypy/jit/backend/test/test_random.py): Fix. ...15:28
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) left irc: Ping timeout: 260 seconds15:44
mvt (~mvt@87.213.45.85) left irc: Quit: Leaving15:49
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) joined #pypy.15:50
Trundle (~andy@p5B14D060.dip.t-dialin.net) joined #pypy.15:58
amaury_>>> help(str.find)15:59
amaury_find(_1, _2, _3=0, _4=2147483647) unbound __builtin__.str method15:59
exarkundelicious16:00
ronny_wow16:01
amaury_ah, it's because it's implemented as a multimethod16:03
amaury_"normal" builtin methods display a better message16:04
amaury_like help(dict.fromkeys):16:04
amaury_fromkeys(type, keys, fill=None) method of __builtin__.type instance16:04
amaury_uh, so dict is a "__builtin__.type instance"16:04
amaury_that's technically true, but useless16:09
xorAxAxhelp(dict.fromkeys, mathematicians_answer=False)16:10
ErKa (~keryell@173-145-39-188.pools.spcsdns.net) joined #pypy.16:13
panni_ (hannes@ip-95-222-52-93.unitymediagroup.de) left irc: Read error: Connection reset by peer16:22
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) left irc: Read error: Operation timed out16:22
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) joined #pypy.16:24
nsf (~nsf@jiss.convex.ru) left irc: Quit: WeeChat 0.3.216:31
elmom_ (~elmom@vallila-gw.hupnet.helsinki.fi) left irc: Read error: Operation timed out16:37
elmom_ (~elmom@vallila-gw.hupnet.helsinki.fi) joined #pypy.16:39
fijalhi16:40
Shanita (~John@g227200134.adsl.alicedsl.de) left irc: Read error: Connection reset by peer16:42
elmom_ (~elmom@vallila-gw.hupnet.helsinki.fi) left irc: Ping timeout: 248 seconds16:44
amaury_fijal: hi16:44
Moku (~John@g227200134.adsl.alicedsl.de) joined #pypy.16:45
fijalpickle error?16:46
fijalhttp://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/325/steps/shell_2/logs/stdio16:46
Alex_Gaynor (~alex@76-206-247-179.lightspeed.cicril.sbcglobal.net) left irc: Ping timeout: 252 seconds16:48
bigdog (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) joined #pypy.16:48
gutworth (~benjamin@64-131-1-181.usfamily.net) joined #pypy.16:57
Alex_Gaynor (~alex@76-206-247-179.lightspeed.cicril.sbcglobal.net) joined #pypy.16:57
arigatofijal: I thought I fixed this one17:03
fijaloh, maybe17:04
fijalthere is a bit of difference between buildbots & my wakeup17:04
fijalhum17:04
fijalbut I did not see any checkin between the run and now17:05
fijalor which revision should have fixed that?17:05
arigato:-(17:06
bitmonk (~jryan@adsl-75-7-0-54.dsl.pltn13.sbcglobal.net) joined #pypy.17:08
Action: gutworth sighs http://mail.python.org/pipermail/python-ideas/2010-June/007328.html17:14
Alex_Gaynorgutworth: you might want to point out that copy.deepcopy has it's own hack implementation of identity dict17:14
exarkunI'm pretty sure Twisted has at least one implementation of the same idea17:16
exarkunIt's sort of hard to grep for though17:16
gutworthAlex_Gaynor: thanks! right in our backyard :)17:16
arigatogutworth: sigh indeed17:17
fijalgutworth: did you point raymond it's not about pypy, it's about any moving gc?17:17
fijalalso, people seem to like AP's idea17:17
fijalof function-based dict17:17
fijalas a factory17:17
gutworthpickle, too17:18
gutworthtes17:18
gutworthyes17:18
arigatogutworth: you might also point out that his solution that seem so obvious to him, namely d[id(obj)], is actually broken17:18
gutworthhow so? if another object is allocated?17:19
arigatoyes17:19
arigatothen you can have a different object with the same id17:19
gutworthwell, a dead object and a live one :)17:19
jcreighwait, what? I thought ids were supposed to be unique for all live objects17:19
gutworth(a bit like social security numbers here :P)17:19
arigatoyes17:19
exarkunjcreigh: But if you don't keep the original key around, only its id, then you don't keep the key alive.17:19
arigatobut the "solution" of using d[id(obj)] is very dangerous17:20
jcreighexarkun: oh!17:20
exarkunhttp://pastebin.com/y6cKXvXc17:20
arigatobecause if obj is deallocated and another obj happens to have the same id17:20
arigatothen d[id(obj)] will succeed by mistake17:20
jcreighugh. You could work around by have the value actually be (obj, value) but nasty.17:21
arigatoyes, I think that the real issue here is that because we don't have an identity dict in the stdlib, it's tempting to "just" use id as Raymond does, and then we get nasty bugs17:22
elmom_ (~elmom@hoasnet-fe29dd00-137.dhcp.inet.fi) joined #pypy.17:24
jcreighhow do you do an efficient identity dict with a moving GC anyway? Do you have to have a special hook to "rehash" the dict after a GC run?17:26
gutworthah, twisted/persisted/styles.py17:26
gutworthjcreigh: we use the interpreter level hash17:26
gutworthwhich is tagged onto the object when its moved17:26
gutworthit's17:26
jcreighah17:26
Alex_Gaynorwould it be worht it to maybe just use an extra word in the obj header for storing id?17:27
exarkunAlex_Gaynor: Worth what?  Avoiding continuing to argue with raymond?17:27
fijalAlex_Gaynor: that's an extra word in the obj header17:27
fijalwhich means even if you don't use id, you pay for it quite a bit17:27
arigatoAlex_Gaynor: see http://morepypy.blogspot.com/2009/10/gc-improvements.html17:28
Alex_Gaynoryes, was just thinking outloud :)17:28
arigatoAlex_Gaynor: in particular, id() is discussed in the comments17:28
amaury_gutworth: did someone mention that an identity_dict is both faster and smaller than a dict with key=id(obj)?17:28
gutworthraymond's contention is that it wouldn't be17:29
amaury_smaller because in most cases, you need to keep to keep the object alive17:29
amaury_so d[id(obj)] = (obj, value)17:29
amaury_which adds a 2-tuple for every entry17:30
amaury_this is how pickle uses it17:32
Alex_Gaynoramaury_: copy does it different, it does d[id(o)] = value; and then d[id(d)].append(o)17:33
Alex_Gaynoryay for consistancy17:33
ErKa (~keryell@173-145-39-188.pools.spcsdns.net) left irc: Ping timeout: 260 seconds17:33
fijalgutworth: I think it's worth pointing out to raymond how different places in stdlib uses different half-broken solutions17:33
arigatohttp://codespeak.net:8099/summary?category=jit  -- oups17:33
fijalon the other hand, it's unclear to me whether raymond would listen17:34
fijalSEGFAULT?17:34
arigatoyes17:34
gutworthfijal: doing that know17:35
gutworththanks :)17:35
amaury_we could suggest the identity_dict idea to the unladen-swallow team, they already did some improvement to cPickle17:37
arigato:-)17:37
fijalas a mean of bypassing raymond?17:37
arigatonot to mention (again) that the performance of id() sucks completely on Jython and IronPython17:37
arigato(and on pypy)17:37
fijalor on cpython if it ever moves to a different gc17:38
Alex_Gaynorfijal: so in 2020 ;)17:38
amaury_hum, I don't see how a PyObject* could cope with a moving gc17:38
arigatofijal: it will never do that17:39
fijalUS had it in their goals17:39
gutworthin 2020, we'll all be coding with PyPy++17:39
gutworth(yes, past tense)17:40
Alex_Gaynorfijal: backwards compatibility is a bitch ;)17:40
arigatofijal: I suppose that r75002 is guilty, as don't see anything else17:41
arigatoand in the blackhole-improvement branch, another obscure failure:17:42
arigatohttp://codespeak.net:8099/builders/jit-benchmark-linux-x86-32/builds/324/steps/shell_2/logs/stdio17:42
ambroff (~ambroff@65-113-99-90.dia.static.qwest.net) joined #pypy.17:48
fijalarigato: :(17:49
SandGorgon (~OmNomNomO@122.162.129.207) left irc: Ping timeout: 245 seconds17:54
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) left irc: Ping timeout: 260 seconds17:59
Ademan_ (~dan@adsl-75-62-232-140.dsl.pltn13.sbcglobal.net) joined #pypy.17:59
ErKa (~keryell@173-145-39-188.pools.spcsdns.net) joined #pypy.17:59
ronny_Alex_Gaynor: anyvc now has some kind of internal painfull checkout/clone api, i hope i have a good idea for a nicely usable public one till next week17:59
Ademan (~dan@adsl-75-36-244-198.dsl.pltn13.sbcglobal.net) left irc: Ping timeout: 276 seconds18:02
Alex_Gaynorronny_: cool FWIW if you'r elooking for a test bed it might be fine to port pip to use it, to make sure the APIs are nice18:03
ronny_Alex_Gaynor: i dont think they want dependencies18:04
ronny_Alex_Gaynor: alos i just started to adapt py.path in anyvc18:04
Alex_Gaynorronny_: probably not, but just as a proof of concept use of the API :)18:04
amaury_ (~5bc3480e@gateway/web/freenode/x-uurjjagxlafxaxtz) left irc: Quit: Page closed18:11
e-max_ (~e-max@217.14.244.16) left irc: Ping timeout: 245 seconds18:11
ErKa (~keryell@173-145-39-188.pools.spcsdns.net) left irc: Ping timeout: 276 seconds18:14
jcreighso does every guard have a matching faildescr instance?18:15
gutworthyep18:15
fijalgutworth: so how about having a factory like r_dict?18:24
fijalketfuncdict18:24
fijalor something18:24
gutworthyep18:24
Alex_Gaynorfijal: what would be the func for pypy?18:24
fijalI don't really care18:25
Alex_Gaynorwell I'm just saying if the issue is to the implement identity_dict as an abstraction, there's no python level func that helps us is there?18:25
fijalif we create a factory18:29
fijalthen we don't really care, right?18:29
fijalbecause we can return a different implementation18:29
Alex_Gaynorfijal: ah you're suggesting that key_funcdict(id) would magically return an identity_dict?18:29
fijalyeah18:30
fijalnot magically18:30
fijalvia return18:30
Alex_Gaynorhaha, I just meant it would be a special case18:30
ErKa (~keryell@173-140-220-61.pools.spcsdns.net) joined #pypy.18:31
nsf (~nsf@jiss.convex.ru) joined #pypy.18:33
jcreighhmm, for the JIT, we don't make any efforts to combine floats in the float constant pool?18:36
arigatoindeed18:37
CIA-803arigo 07roundup * 10#544/AssertionError: [new] With trunk, no jit: arigo@tannit32 ~/svn/pypy/benchmarks/unladen_swallow/performance $ ~/pypysrc/32compiled/pypy-c-75020 bm_rietveld.py RPy ... * 14https://codespeak.net/issue/pypy-dev/issue54418:44
ErKa (~keryell@173-140-220-61.pools.spcsdns.net) left irc: Ping timeout: 240 seconds18:48
fijalarigato: I would suspect recent changes to pickling18:49
fijalto accomodate stackless18:49
fijalor uh18:49
fijalmaybe not18:49
fijalwe don't know when it happened because trunk was broken on benchmarks for a bit18:49
arigatofijal: I'm not looking at it, I'm looking at a different error18:51
arigatoI've posted it there to not forget18:51
arigatowell I'm looking at two different unrelated errors already18:51
fijalI got swamped with unrelated errors recently18:54
fijaland now I'm untangling encoding freezing to be able to debug a real error18:54
fijal:-/18:54
kursor (~kursor@p57BB236B.dip0.t-ipconnect.de) left irc: Quit: kursor18:57
arigato:-)19:02
fijalwell19:02
fijalthat's not exactly an emoticon I would use19:02
jcreighIn the encoding produced by write_failure_recovery_description, what is the purpose of CODE_HOLE?19:05
ErKa (~keryell@173-141-50-250.pools.spcsdns.net) joined #pypy.19:08
arigatofijal: yes, sorry19:14
ErKa (~keryell@173-141-50-250.pools.spcsdns.net) left irc: Ping timeout: 276 seconds19:17
kenaan03arigo 10r7502314 pypy/trunk/pypy/jit 07M(model.py assembler.py regalloc.py pyjitpl.py): Revert r75002, r75004 and r75012.  Until I find out why, ...19:17
fijalshall I kick buildbot?19:18
arigatoif you feel like it, yes19:19
Action: arigato is thinking about writing code to check that registers usage is valid in the jitcodes in the blackhole-improvement branch19:19
ErKa (~keryell@173-141-50-250.pools.spcsdns.net) joined #pypy.19:20
Action: fijal feels like it19:22
fijalprobably makes sense19:22
fijalonce we have register allocator, we can use it to have a pure-assembler static backend, right?19:23
arigatoyes, in theory19:23
dmalcolm (~david@nat/redhat/x-sjoywobqhjipgvrv) joined #pypy.19:23
fijalin theory, given enough work, it should be faster than C backend19:23
arigatowell the allocator doesn't care about "but this machine has only 6 registers"19:23
fijalin practice, we can never achieve that19:24
gutworthlet's not rewrite the *whole* toolchain19:25
gutworthbefore you know it we'll make people use our own cpus...19:25
arigatoyes :-)19:25
exarkundoes clang build what pypy's c backend emits?19:25
arigatoI suppose, it's ANSI C19:25
Alex_Gaynorexarkun: no, since we post process ASM19:25
arigatoah19:26
Alex_Gaynorso yes I suppose clang builds it19:26
gutworthAlex_Gaynor: well, asmgcc root won't work19:26
Alex_Gaynorbut we can't post process the ASM19:26
gutworthshadow stack19:26
Alex_Gaynorgutworth: are you volunteering to write another ASM parser?19:26
arigatosome time ago I would have asked if clang could have the feature or be easily extensible to do directly our assembler-parsing hack19:27
procto (~boo@logarchy.org) left irc: Ping timeout: 276 seconds19:27
gutworthno, I'm volunteering you to write a clang plugin19:27
arigatobut now I don't because it would be yet another mess with the JIT19:27
Alex_Gaynorguibou: speaking of which (not really), when do you start work on 2.6?19:28
Alex_Gaynorer gutworth 19:28
gutworth22 hours19:29
arigato:-)19:29
ErKa (~keryell@173-141-50-250.pools.spcsdns.net) left irc: Ping timeout: 260 seconds19:30
mivert_ (~mivert@208.78.39.48) joined #pypy.19:30
Alex_Gaynorgutworth: cool, I may volunteer to write an rpython json module, it sounds like fun19:30
xorAxAxwe are the knights who say NIH, we want an json shrubbery19:31
exarkunrpython parser generator with json grammar as input19:31
exarkungutworth: didn't you write an rpython parser generator?19:31
gutworthdoes json have grammar?19:31
jcreighhaha, yes19:31
jcreighI mean, it's not a regular language19:32
gutworthexarkun: I did19:32
Alex_Gaynorhttp://json.org/19:32
exarkungutworth: what kind of languages does it support?19:32
gutworthas long as its LL119:32
gutworthgerman and afrikans19:32
Alex_GaynorJSON is weaker than LL(1) I think19:32
Alex_Gaynora stack machine can handle it I think19:32
exarkunAlex_Gaynor: So there you go.  Just use what gutworth wrote.19:33
gutworthah, they have a nice lexer grammar on the right19:33
Alex_Gaynorexarkun: but that's not as awesome!19:33
exarkunAlex_Gaynor: It's more awesome.19:33
exarkunAlex_Gaynor: Because it avoids wasting time and writing unnecessary code19:33
Alex_GaynorI hadn't considered that19:33
mivert (~mivert@208.78.39.48) left irc: Ping timeout: 258 seconds19:33
Action: jcreigh wishes he could wave a magic wand and replace 99% of the uses of XML with JSON19:36
Alex_Gaynorjcreigh: my boss was a part of the original XML comitee, I'll never let him live it down :)19:36
exarkunjcreigh: man don't waste wishes or magic wands like that :/19:36
arigatowaa19:44
arigatothe jitcode is so wrong that I can see it just by a quick look at its dump19:44
Alex_Gaynorarigato: I'm not sure if that is better or worse than it being only subtly broken19:45
arigatothat's unclear if it's really the issue there, so more staring is needed :-(19:45
procto (~boo@logarchy.org) joined #pypy.19:48
ErKa (~keryell@173-114-2-93.pools.spcsdns.net) joined #pypy.19:50
fijalarigato: you mean reg alloc?19:50
arigatofijal: no, some other detail19:50
arigatoas a side note, no jitcode uses more than 12 ref registers, 14 int registers, or 6 float registers19:51
arigatomaybe allocating 256 of each is a bit too much19:52
fijalhow often do we allocate them?19:53
arigatonot often19:53
fijalwell, it seems that doesn't matter19:54
arigatoyes19:54
fijalsince we get better cache locality19:54
fijalif we use smaller tables19:54
arigatono, even not19:54
fijalbecause?19:54
arigatocache locality is not really about using memory that is close together19:54
arigatoit's about not using a lot of memory in total19:55
arigato(well it's about using memory that is very close in the sense of "in the same row of 32 bytes" or something)19:56
fijalk19:56
fijaldid you get my mails btw?19:56
arigatoyes19:57
fijalany opinion on number?19:59
Ademan_ (~dan@adsl-75-62-232-140.dsl.pltn13.sbcglobal.net) left irc: Ping timeout: 260 seconds19:59
fijalon 'def number'19:59
Ademan (~dan@adsl-75-62-234-243.dsl.pltn13.sbcglobal.net) joined #pypy.19:59
fijalok, ignore it, you have other problems20:00
arigatowhat opinion could I have?  we know that a lot of time is spent in resume.py in general20:04
arigatoI didn't know def number was so high20:04
arigatoI don't know what to do20:05
arigatomaybe we could replace dictionaries with fields in their key objects20:06
arigatoif we are very careful20:06
fijal (~fijal@2002:47c4:f79c:0:218:deff:fe73:90b1) left irc: Read error: Operation timed out20:07
ErKa (~keryell@173-114-2-93.pools.spcsdns.net) left irc: Ping timeout: 248 seconds20:16
ErKa (~keryell@173-114-2-93.pools.spcsdns.net) joined #pypy.20:23
bigdog (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) left irc: Quit: Leaving.20:27
fijal (~fijal@rtr-folsom.wireless.indra.com) joined #pypy.20:29
fijalre-hi20:29
fijalarigato: my point was more than number() takes more time than assembler generation for example20:30
fijaland I don't completely understand why20:31
fijalthere is a couple of comments like "XXX would be nice to have this or that"20:31
fijalwhich might improve situation drastically20:31
ErKa (~keryell@173-114-2-93.pools.spcsdns.net) left irc: Ping timeout: 276 seconds20:36
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) joined #pypy.20:49
kenaan03fijal 10r7502414 05A(pypy/branch/unfreeze-encodings/): A branch to try to unfreeze encodings ...20:49
bigdog (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) joined #pypy.21:00
copelco_ (~copelco@h-66-166-135-210.atlngahp.static.covad.net) joined #pypy.21:02
amaury_ (~5645c527@gateway/web/freenode/x-lghdhszgvrrvgtwi) joined #pypy.21:03
HenrikV (~henrikven@cpc1-acto1-2-0-cust209.4-2.cable.virginmedia.com) left irc: Ping timeout: 245 seconds21:04
HenrikV (~henrikven@cpc1-acto1-2-0-cust209.4-2.cable.virginmedia.com) joined #pypy.21:05
copelco (~copelco@h-66-166-135-210.atlngahp.static.covad.net) left irc: Ping timeout: 258 seconds21:06
copelco_ (~copelco@h-66-166-135-210.atlngahp.static.covad.net) left irc: Ping timeout: 245 seconds21:06
AndrewBC (MGM@97.93.232.247) joined #pypy.21:10
HenrikV (~henrikven@cpc1-acto1-2-0-cust209.4-2.cable.virginmedia.com) left irc: Ping timeout: 245 seconds21:19
fubd (~a@digsby05.rit.edu) left irc: Ping timeout: 276 seconds21:22
romao (~Caio@unaffiliated/romao) left irc: Quit: nil21:25
Arfrever (~Arfrever@gentoo/developer/Arfrever) left irc: Quit: Ex+re (KVIrc 4)21:26
gonsor (~quassel@kel30.kel.stud.uni-goettingen.de) left irc: Ping timeout: 260 seconds21:27
fubd (~a@digsby05.rit.edu) joined #pypy.21:28
kenaan03getxsick 10r7502514 06D(pypy/branch/fast-ctypes/pypy/module/jitffi/): kill pypy/module/jitffi for now ...21:28
Shanita (~John@f048113157.adsl.alicedsl.de) joined #pypy.21:30
kenaan03getxsick 10r7502614 pypy/branch/fast-ctypes/pypy/rlib 05A(jitffi.py test_jitffi.py): add pypy/rlib/jitffi module ...21:30
Moku (~John@g227200134.adsl.alicedsl.de) left irc: Read error: Operation timed out21:31
kenaan03getxsick 10r7502714 07M(pypy/branch/fast-ctypes/pypy/rlib/test/test_jitffi.py): remove comments ...21:31
fijalarigato: http://buildbot.pypy.org/builders/jit-benchmark-linux-x86-32/builds/326/steps/shell_5/logs/stdio21:34
fijalgood :)21:34
fijalin a way21:34
ousado_ (~ousado@p5B3C0CC2.dip0.t-ipconnect.de) joined #pypy.21:34
fijalon the other hand it's still broken in a way21:36
ousado (~ousado@p5B3C2A1D.dip0.t-ipconnect.de) left irc: Ping timeout: 245 seconds21:37
gutworthhaha, that avoids our bugs well21:42
ErKa (~keryell@173-114-2-93.pools.spcsdns.net) joined #pypy.21:43
fijalgutworth: what avoids our bugs?21:46
gutworthother failing software21:49
fijaleh21:52
arigatofijal: yes... I guess I will make a branch, re-apply there, and debug21:56
fijalarigato: we have at least 3 different problems here21:57
arigatoyes21:57
fijal1) templess not resolbving21:57
fijal2) speed not working21:57
fijal3) your issue21:57
fijaland 4) my issue21:57
fijalwe have 4 different problems...21:57
fijalwhat about templess?21:57
fijalwill you fix it or should we disable this step?21:57
arigatotempless?21:57
arigatoah21:57
arigatoI already made a local copy in the repo21:58
arigatoI guess I forgot to redirect21:58
kenaan03afa 10r7502814 05A(pypy/trunk/pypy/module/cpyext/patches/boost.patch): The patch I use to compile Boost.Python. ...21:58
arigatofijal: it's in http://codespeak.net/svn/user/arigo/hack/pypy-hack/templess21:59
fijalwill you change?22:00
arigatohttp://paste.pocoo.org/show/221208/22:01
arigatois it enough?  when run locally it complain that 'templess' is already checked out from another url,22:02
arigatobut I guess it does not matter in any automated test, right?22:02
fijalshould not22:04
fijalsince it removes it (I think)22:04
fijalwell, there is also a question whether you want to keep your old runs around22:04
Alex_Gaynorfijal: what is the argument to valgrind to use callgrind?22:06
fijalvalgrind --tool=callgrind22:07
kenaan03arigo 10r7502914 07M(pypy/trunk/pypy/translator/benchmark/benchmarks.py): Use our local copy of templess. ...22:07
jcreighhmm, what sorts of functions does the JIT call anyway? (from, eg, genop_call in assembler.py). I'm assuming that Python-level function calls are invisible to the JIT, so are these RPython functions we're calling?22:07
arigatojcreigh: yes22:08
arigatoso genop_call and the CALL resoperation must produce an assembler call that follows the calling conventions22:08
jcreighyeah22:09
arigato(these RPython functions are of course turned into C and compiled, at that point in time)22:09
arigato(unless we are running tests, in which case we use a ctypes callback)22:09
fijalarigato: can you answer my quesiton? (whether it still makes sense to run your old benchmark suite)22:11
kenaan03afa 10r7503014 05A(pypy/branch/cpyext-init-cleanup/): A branch to clarify the mess in the different objects exposed by the API ...22:11
arigatofijal: sorry, what do you mean?22:12
fijaltempless is run within the old benchmark22:12
fijaloutside completely of what speed sees22:12
fijalyou made a backend using gnuplot a while ago22:12
fijaldo you still use it?22:12
Nick change: DasIch_ -> DasIch22:12
arigatoah, no22:12
fijalif not, then we can remove a whole lot of code :)22:13
kenaan03fijal 10r7503214 07M(pypy/build/bot2/pypybuildbot/builds.py): These days we have only one way to run benchmarks ...22:14
fijalI can clean this up then22:14
fijalsince we have quite a bit of support here and there in pypy for running a variety of benchmarks22:14
fijalarigato: from my measurments it seems we can cut down time of jitting quite a bit22:15
fijalthat would still not really speed up translation that much22:15
mwhudsoni wonder if tuatara is still failing to build pypy every night22:19
lucian (~lucian@94-193-66-109.zone7.bethere.co.uk) joined #pypy.22:19
fijalmwhudson: interesting question22:19
Alex_Gaynorfijal: hrm, trying to figure out why this one test in django's test suite is so slow on pypy, no DB access in the test itself :/22:22
fijalfigure it out :)22:22
Alex_GaynorI'm trying, but there's nothing obvious in the callgrind profile22:23
kenaan03fijal 10r7503314 07M(pypy/branch/unfreeze-encodings/pypy/module/_codecs/interp_codecs.py): This is the state I would like to achieve (breaks translation) ...22:23
fijalAlex_Gaynor: was it slow under valgrind, or just slow?22:23
fijaland how much slower?22:23
fijalcan you paste me your session which showcases it's slow and then send me callgrind profile?22:23
Alex_Gaynorunder pypy it is about 3.3s vs. .8s on cpython22:23
Hory (~hory@78.96.96.85) left irc: Quit: Leaving.22:23
Alex_Gaynorsure I can make a callgrind profile22:23
fijalI would probably not tell you much, but please do :)22:23
fijalmwhudson: don't learn about django too much22:26
mwhudsonfijal: why?22:26
fijalit's terrible on quite some layers22:26
fijaldoctests comes to mind22:26
fijalinherently slow22:26
Alex_Gaynorfijal: we're fixing them!22:26
mwhudsonfijal: launchpad is zope3, i bet django isn't more inherently slow than that22:27
fijalAlex_Gaynor: you're not fixing lack of unittests22:27
Alex_Gaynorit's true most of our tests are functional/integration22:27
fijalmwhudson: fair enough. I compare django against my mythical web framework that does not exist22:27
mwhudsonfijal: right, they all suck22:27
fijala deep temptation to hide errors from you22:27
fijaland instead show you something irrelevant22:27
jcreighOTOH, Django works if you want to build websites...22:28
fijaldjango is good at catching precise errors and generalizing them to garbage22:28
Alex_Gaynorfijal: is this supposed to be like 30x slower under callgrind?22:28
fijalAlex_Gaynor: yeah22:28
Alex_Gaynorok actually 60x slower :D22:28
fijalcallgrind is a cpu emulater22:28
Alex_Gaynorfijal: where do you want me to send these callgrind files (I got 5 of these)22:29
fijalonly one of them is non-922:29
fijalnon-022:29
fijalI bet22:29
fijalto my mail?22:29
Alex_Gaynorah yes22:29
Alex_Gaynorfijall@gmail?22:29
Alex_Gaynorfijal: sent22:30
Alex_Gaynorit looks like too mcuh time is spent in JIT actually22:30
fijalwhat's the JIT report at the end?22:31
Alex_Gaynoroptimizer and resume data are the highest functions22:31
Alex_GaynorI'm compiled with --jit-debug=off22:31
Alex_Gaynorwant me to compile with it on?22:31
fijalwould be cool22:32
fijalyou can also overload this option22:32
fijalfrom command line22:32
ErKa (~keryell@173-114-2-93.pools.spcsdns.net) left irc: Ping timeout: 240 seconds22:32
fijalalso you can try PYPYLOG=log pypy-c ....22:32
Alex_Gaynorok I'll do that as well22:32
fijaland then analyze the log using pypy/tool/logparser.py22:32
fijalit does not look that bad22:33
fijalnote - it's about 30% in JIT22:33
kenaan03getxsick 10r7503414 pypy/branch/fast-ctypes/pypy/rlib 07M(jitffi.py test_jitffi.py): add the Get class. ...22:33
fijalwhich still leaves room for improvements somewhere else22:33
fijalI don't know, slice?22:33
Alex_Gaynor?22:33
fijalslice the problem into pieces22:34
Alex_Gaynoryes according to this 40% execution, 30% jit, 17% gc, and 17% jit-running22:34
fijal(have fun doing this)22:34
fijalso well22:34
fijalnot enough for JIT to kick in22:34
fijalI don't care then too much22:34
Alex_Gaynorthere's 12% blackhole as well22:35
kenaan03getxsick 10r7503514 07M(pypy/branch/fast-ctypes/pypy/rlib/jitffi.py): kill imports ...22:35
Alex_Gaynoralmost as much blackhole as jit running22:35
Kaskuka (~John@f054209159.adsl.alicedsl.de) joined #pypy.22:35
Shanita (~John@f048113157.adsl.alicedsl.de) left irc: Ping timeout: 260 seconds22:36
Alex_Gaynorfijal: it's 3x slower than cpython, there's got to be something that can be done22:36
fijalhow about trying blackhole-improvements branch?22:37
gutworthrewrite it in cobol22:37
fijalwell, also run this test a couple of times in a row22:37
fijaland see if this is jit-warmup issue22:38
bitmonk (~jryan@adsl-75-7-0-54.dsl.pltn13.sbcglobal.net) left irc: Quit: bitmonk22:38
bitmonk (~jryan@adsl-75-7-0-54.dsl.pltn13.sbcglobal.net) joined #pypy.22:38
Alex_Gaynorok going to run it 5x22:39
Alex_Gaynorfijal: running it 10x results in 17s vs. 6s22:42
fijaleh22:42
fijalI hate your approach to benchmarking22:42
Alex_Gaynorhow do you mean?22:43
fijalshow info for each run22:43
fijalto see how the warmup time distributes22:43
fijalthe OMG IT'S SLOW22:43
fijalapproach22:43
Alex_Gaynorhehe fair enough, I'll get more precise benchmarks going, but first a break to read for a bit22:44
izq (~izq@pcsp232-243.dynamic.supercable.net.ve) joined #pypy.22:47
fijalhttp://paste.pocoo.org/show/221227/22:53
fijalgreat22:53
kenaan03fijal 10r7503614 07M(pypy/branch/unfreeze-encodings/pypy/module/_codecs/interp_codecs.py): Remove some nonsense. call_function will raise correct exception if asked, it's ...22:54
gutworthnice22:58
fijalour codecs are a mess :-/22:58
ly- (~ly-@unaffiliated/ly-) left irc: Quit: ly-23:01
AndrewBCAccording to the title of this page: http://codespeak.net/svn/greenlet/trunk/doc/greenlet.txt greenlets are concurrent, but I've been reading and thinking, and it seems to me that given the methods presented there, the only way to get concurrency is to have greenlets in different threads, which is mentioned23:05
AndrewBCam I right in that, or am I missing something?23:05
gutworthwell, yes, it would have to be implemented with threads on the underlying os23:05
AndrewBCOkay :)23:06
kenaan03fijal 10r7503714 07M(pypy/branch/unfreeze-encodings/pypy/objspace/std/unicodetype.py): A shortcut for decoding common cases ...23:06
arigato (~arigo@c-354be353.022-54-67626719.cust.bredbandsbolaget.se) left irc: Quit: See you23:07
exarkun'pypy-c setup.py install' will put extension modules into a place where they can be imported by pypy-c later, right?  And what's the default location?23:08
AndrewBCI'm just messing around making a simple server that spins off newly accepted connections to greenlets, in case you're interested (though I'm sure this has been done a million times)23:08
fijalexarkun: anto is trying to solve this out23:09
fijalas of now "I don't even want to know"23:09
fijalbasically where distutils think it's a good idea23:10
exarkunI suppose I can pass a --prefix23:10
Action: exarkun tries to figure out how to get buildbot to build extensions with pypy so they'll be available for use by later test suites23:14
kenaan03fijal 10r7503814 07M(pypy/branch/unfreeze-encodings/pypy/objspace/std/unicodetype.py): Write down performance shortcuts for encode case ...23:16
fijal(Pdb) encoding23:19
fijal'ANSI_X3.4-1968'23:19
fijalwhat's that?23:19
mwhudson (~mwh@canonical/launchpad/mwhudson) left irc: Ping timeout: 264 seconds23:19
izq (~izq@pcsp232-243.dynamic.supercable.net.ve) left irc: Ping timeout: 240 seconds23:19
exarkunIt's an alias for something a lot more obvious23:19
fijalexarkun: any clue?23:19
fijallike ascii?23:19
exarkunYes, it seems23:20
exarkunhttp://en.wikipedia.org/wiki/ASCII23:20
fijal_getfilesystemencoding returns that23:22
fijalawesome23:22
Shinato (~John@f048098204.adsl.alicedsl.de) joined #pypy.23:24
Kaskuka (~John@f054209159.adsl.alicedsl.de) left irc: Ping timeout: 276 seconds23:24
kenaan03fijal 10r7503914 07M(pypy/branch/unfreeze-encodings/pypy/module/sys/interp_encoding.py): Obscure specialcase ...23:25
gutworthfijal: you realize that people can set the encoding to anything defeating any shortcuts you might have, right?23:29
fijal (~fijal@rtr-folsom.wireless.indra.com) left irc: Ping timeout: 260 seconds23:30
amaury_ (~5645c527@gateway/web/freenode/x-lghdhszgvrrvgtwi) left irc: Quit: Page closed23:31
Moku (~John@e179108246.adsl.alicedsl.de) joined #pypy.23:31
dmalcolm (~david@nat/redhat/x-sjoywobqhjipgvrv) left irc: Quit: Leaving23:32
Shinato (~John@f048098204.adsl.alicedsl.de) left irc: Ping timeout: 252 seconds23:33
pedronis (~pedronis@46-180.61-188.cust.bluewin.ch) left irc: Ping timeout: 258 seconds23:38
lucian (~lucian@94-193-66-109.zone7.bethere.co.uk) left irc: Quit: Leaving...23:40
getxsickamazing, my module adds 2 floats with the wrong result23:41
bigdog (~scmikes@72-197-8-8-arpa.cust.cinci.current.net) left irc: Ping timeout: 248 seconds23:42
DasIch (~DasIch@p5DC5EE3E.dip.t-dialin.net) left irc: Quit: Lost terminal23:43
Alex_Gaynorgetxsick: but does it do it quickly?23:45
getxsickenough, so ok right?23:46
Alex_Gaynor:)23:46
getxsickhm23:46
getxsickE       assert 2.7000000000000002 == 1.87523:46
getxsicknot a big deal23:46
getxsickcan i call it a new feature?23:47
izq (~izq@190.39.218.20) joined #pypy.23:47
beutdeuce (~44ae292b@gateway/web/freenode/x-xzhtxdjwriizczpy) joined #pypy.23:55
beutdeuceI'm having trouble getting translatorshell.py working23:55
beutdeuceimport failures23:55
getxsickpastebin the output23:57
getxsickit's hard to predict the failures ;)23:58
beutdeucehttp://pastie.org/98983223:58
getxsickis it trunk checkout or what?23:59
beutdeucepypy.org download for os x 1.223:59
--- Thu Jun 3 201000:00

Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!