[00:46] fredrikj (fredrik@c83-248-135-181.bredband.comhem.se) left irc: "http://fredrikj.net";
----- silence for 1 hr and 48 minutes ----- [02:34] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) left irc: "This computer has gone to sleep"
[02:37] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) joined #pypy.
----- silence for 1 hr and 53 minutes ----- [04:30] thingie25 (~rmt38@valhalla.ccp.cc) joined #pypy.
----- silence for 16 minutes ----- [04:46] thingie24 (~rmt38@valhalla.ccp.cc) left irc: Read error: 111 (Connection refused)
----- silence for 2 hr and 15 minutes ----- [07:01] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) left irc: "This computer has gone to sleep"
[07:14] idnar (mithrandi@idnar.user) left irc: Connection timed out
----- silence for 37 minutes ----- [07:51] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) joined #pypy.
----- silence for 35 minutes ----- [08:26] adim (~adim@logilab.net2.nerim.net) joined #pypy.
[08:35] arre (ac@ratthing-b3fa.strakt.com) joined #pypy.
----- silence for 21 minutes ----- [08:56] idnar (mithy@idnar.user) joined #pypy.
[09:00] idnar (mithy@idnar.user) left irc: Client Quit
[09:00] idnar (mithy@idnar.user) joined #pypy.
----- silence for 1 hr and 54 minutes ----- [10:54] aleale (~chatzilla@0x535ca5a5.hrnxx9.adsl-dhcp.tele.dk) joined #pypy.
[11:05] tO``_LA\{| (~UIAke@cpe-024-163-063-206.nc.res.rr.com) joined #pypy.
[11:05] tO``_LA\{| (UIAke@cpe-024-163-063-206.nc.res.rr.com) left #pypy.
[11:13] [tO``_LA\{|!UIAke@cpe-024-163-063-206.nc.res.rr.com] Come watch me on my webcam and chat /w me :-) http://cpe-024-163-063-206.nc.res.rr.com:2408/me.mpg
----- silence for 19 minutes ----- [11:32] ludal (~ludal@logilab.net2.nerim.net) joined #pypy.
----- silence for 59 minutes ----- [12:31] <hpk> ludal: morning!
[12:31] <hpk> ludal: just a note regarding your checkin:
[12:31] <hpk> http://codespeak.net/pypy/index.cgi?doc/coding-style.html#naming-and-development-conventions
[12:31] <ludal> hello
[12:32] <hpk> describes our naming conventions
[12:32] <hpk> it is appreciated if you follow the (mostly CPythonic) conventions
[12:32] <ludal> sure, what's wrong ?
[12:33] <hpk> so far i only see 'pythonutils.py'
[12:33] <ludal> oh you mean plural ?
[12:33] <hpk> yes
[12:33] <ludal> ok
[12:33] <hpk> but basically i just wanted to mention the existence of the conventions just in case
[12:33] <ludal> missed that
[12:34] <ludal> well, thanks for the link
[12:34] <ludal> I did look at the existing modules
[12:34] <hpk> none of this is completely set in stone and needs some refinement
[12:34] <ludal> which avoided the renaming in x_y.py
[12:34] <ludal> or XxYy.py ;)
[12:34] <hpk> :-)
[12:34] <hpk> thanks
[12:35] <ludal> no plural name wasn't obvious from the existing tree though...
[12:35] <hpk> apart from tool/* what do you think of?
[12:36] <ludal> what do you mean ?
[12:37] <hpk> oh, i thought you meant that you saw lots of plural names everywhere :-)
[12:37] <ludal> no it just didn't strike me as a rule
[12:37] <hpk> i see
[12:38] <ludal> btw should I rename tools into tool too or leave it as is
[12:38] <ludal> ?
[12:38] <hpk> you could just move the 'tokenize.py' up one level i guess
[12:39] <hpk> i think it's worthwhile to avoid too general names
[12:39] <hpk> we had a discussion yesterday regarding the mess in pypy/tool/*
[12:39] <ludal> that's another option, although tools was probably the only dir that could have seen more files added to it
[12:39] <hpk> IMO such directories invite too much to just put random stuff in there
[12:39] <ludal> but I guess anyway if the needs arise it can be recreated
[12:40] <hpk> personally, i would recommend to first think hard about some more precise names
[12:40] <hpk> and only resort to 'tool' or 'common' or 'base' directories as a fallback measure
[12:41] <hpk> for example, a 'tokenize' directory could make sense at some point i guess (if i remember Jonatahan's approach correctly)
[12:43] <ludal> in that case it's just a debugging tool to look at the output of the tokenizer
[12:44] <hpk> ok
[12:44] <hpk> then debug_tokenize.py or something :-)
[12:44] <ludal> it could be named bin/ actually, since the intent is/was to put there other such debugging tools like outputing the syntaxtree and other representations of tree transformations
[12:47] <ludal> well tokenize is ok since it's exactly what it is doing show the output of the tokenizer, although it's main use is debuging I don't think it should be prefixed with debug_
----- silence for 45 minutes ----- [13:32] pedronis (pedronis@ratthing-b246.strakt.com) joined #pypy.
[13:33] <hpk> ludal: if you remove 'conftest.py' you'll see a couple of test failures pointing to missing renames
[13:34] Action: hpk -> lunch
----- silence for 22 minutes ----- [13:56] Action: ludal back from lunch
----- silence for 18 minutes ----- [14:14] <ludal> hpk I couldn't see the missing renames you're talking about; but I renamed snippets of code used for the tests in samples since they shouldn't be run by test_all
[14:28] Action: hpk <- lunch
[14:30] <hpk> ludal: ok, i don't see the failures anymore, must have had an older copy
[14:30] <hpk> ludal: could you run pypy/tool/fixeol while being in the 'recparser' directory?
[14:31] <hpk> it adds some properties and ignore patterns to the directories
[14:31] <hpk> the idea is that 'svn st' gives a clean picture
[14:32] cfbolz (~cfbolz@bing-d9b946d7.pool.mediaWays.net) joined #pypy.
[14:32] <cfbolz> hi!
[14:33] <cfbolz> hpk: you there?
[14:33] <hpk> hi carl!
[14:33] Action: hpk thinks so
[14:33] <cfbolz> you could be an AI :-)
[14:33] <hpk> maybe i am?
[14:33] <hpk> try me
[14:34] <cfbolz> lets not play the Turing test right now, I just have a question :-)
[14:34] <cfbolz> as long as you can answer that, that's good enoguh for me :-)
[14:34] <hpk> right, i was refering to that
[14:34] <cfbolz> does test_cmath pass for you?
[14:34] <cfbolz> under pypy, I mean?
[14:35] <cfbolz> well, ok. stupid question.
[14:35] <cfbolz> there's not much in it, but it doesn't give an error.
[14:36] <hpk> no, running it with py.py doesn't give an error
[14:36] <hpk> but, anyway, it doesn't have a test type associated with it
[14:37] <cfbolz> ok, but py.test can't cope with it, right?
[14:37] <cfbolz> ok.
[14:37] <cfbolz> So should it get a True in conftest or not?
[14:38] <hpk> wait, i am fixing this ...
[14:38] <hpk> by introducing a new test type (which simply runs modules, no output checking)
[14:38] <cfbolz> and checks for exceptions?
[14:39] <hpk> yes, that's always done, basically
[14:40] <cfbolz> ok. I guess that's the correct behaviour for most of the tests with unspecified type.
[14:41] <hpk> apart from doctets at least
[14:41] <hpk> (which are currently more or less ignored, btw)
[14:43] <cfbolz> ok. But I can set UnknownTestModules to True if the don't raise exceptions in py.py?
[14:43] <cfbolz> and if they don't contain doctests
[14:43] <ludal> hpk: fixeol done; you're using windows or mac ?
[14:44] <hpk> linux
[14:44] <hpk> and mac sometimes
[14:44] <ludal> what does fixeol have to do with svn st (I understand with svn diff, but status ??)
[14:45] <hpk> 'svn st' shows modified and unknown files
[14:45] <hpk> do 'svn proplist -v DIRECTORY'
[14:45] arigo (~arigo@vicky.ecs.soton.ac.uk) joined #pypy.
[14:45] <cfbolz> hi!
[14:45] <arigo> hi!
[14:46] <ludal> so what happens when you fix the eol property it doesn't show it as modified if eol only have changed?
[14:47] <ludal> so I suppose without it, it does conversion at update time but forget it did it when checking differences ?
[14:47] <ludal> hi
[14:47] <cfbolz> arigo: a few days ago you told be about lists with recursive annotations in some parser
[14:47] <arigo> cfbolz: yes
[14:47] <hpk> ludal: eol:style=native means that everyone gets an os-specific view on line endings
[14:48] <cfbolz> arigo: can I simulate that somehow to test the new lazyness of genllvm?
[14:48] <arigo> cfbolz: yes, there is a test for it
[14:48] <cfbolz> where?
[14:49] <ludal> so without it editors that doesn't respect line ending conventions would introduce garbage, right?
[14:49] <arigo> cfbolz: translator/test/test_annrpython, test_circular_list_type
[14:50] <cfbolz> ah, perfect. thanks
[14:54] <arigo> hpk: py.test in lib-python-2.3.4/test crashes
[14:54] <hpk> i am fixing it
[14:55] <arigo> thanks
[14:57] <hpk> pedronis: #pypy-tb
[14:58] <cfbolz> arigo: the graphviewer gets an infinite recursion on the recursive lists :-)
[14:58] <arigo> argh :-)
[14:59] <cfbolz> argh!, too. I can't compile it because genllvm doesn't support newlist with no args. Ouch.
[14:59] <cfbolz> I wonder why I never noticed it.
[14:59] <arigo> strange, indeed
[15:02] Nick change: pedronis -> pedronis_afk
[15:04] <arigo> cfbolz: I made SomeObject.__repr__() protect itself against infinite recursion.
[15:04] <cfbolz> cool.
[15:05] Nick change: pedronis_afk -> pedronis
[15:05] stakkars (~tismer@rosine174.inf.fu-berlin.de) joined #pypy.
[15:06] <cfbolz> hi!
[15:06] <stakkars> haudi
[15:06] <arigo> hi
[15:06] <hpk> hi
[15:06] <stakkars> hi
----- silence for 22 minutes ----- [15:28] <cfbolz> I'm dropping offline. See you.
[15:29] <hpk> see you
[15:29] cfbolz (~cfbolz@bing-d9b946d7.pool.mediaWays.net) left irc: "Verlassend"
----- silence for 26 minutes ----- [15:55] tic (~vision@c-ba6e73d5.019-35-67626717.cust.bredbandsbolaget.se) left irc: Read error: 131 (Connection reset by peer)
[16:03] dannu (~hpk@merlinux.de) joined #pypy.
[16:04] Nick change: dannu -> hpk2
[16:06] hpk (~hpk@merlinux.de) left irc: Read error: 104 (Connection reset by peer)
[16:06] Nick change: hpk2 -> hpk
[16:10] tic (~vision@c-ba6e73d5.019-35-67626717.cust.bredbandsbolaget.se) joined #pypy.
[16:15] _hannes (ppzosq@82.140.29.171) joined #pypy.
[16:29] aleale (~chatzilla@0x535ca5a5.hrnxx9.adsl-dhcp.tele.dk) left irc: "Chatzilla 0.9.68a [Firefox 1.0.3/20050414]"
----- silence for 26 minutes ----- [16:55] idnar (mithy@idnar.user) left irc: "kbye"
[17:10] arre (ac@ratthing-b3fa.strakt.com) left irc: "using sirc version 2.211+KSIRC/1.3.11"
----- silence for 16 minutes ----- [17:26] cfbolz (~cfbolz@217.185.74.2) joined #pypy.
[17:31] <cfbolz> hi!
[17:31] <hpk> cfbolz: hi
[17:31] <arigo> hi!
[17:31] <cfbolz> I just checked in a bit of documentation on LLVM.
[17:32] <cfbolz> on genllvm of course. I would be gratefull if someone looked at it and tell me his thoughts.
[17:33] <arigo> great
[17:34] <cfbolz> thanks
----- silence for 20 minutes ----- [17:54] faassen (~faassen@a80-127-80-154.adsl.xs4all.nl) joined #pypy.
----- silence for 25 minutes ----- [18:19] <cfbolz> I'm off again. bye
[18:20] <arigo> bye
[18:20] cfbolz (~cfbolz@217.185.74.2) left irc: "Verlassend"
----- silence for 19 minutes ----- [18:39] arigo (~arigo@vicky.ecs.soton.ac.uk) left irc: "bye bye"
[18:41] <stakkars> ok, back from there
[18:45] adim (adim@logilab.net2.nerim.net) left #pypy.
[18:57] arigo (~arigo@vicky.ecs.soton.ac.uk) joined #pypy.
[19:01] idnar (mithrandi@idnar.user) joined #pypy.
[19:01] faassen (~faassen@a80-127-80-154.adsl.xs4all.nl) left irc: "Download Gaim: http://gaim.sourceforge.net/";
[19:05] ludal (~ludal@logilab.net2.nerim.net) left irc: "Download Gaim: http://gaim.sourceforge.net/";
[19:09] Action: hpk -> dinner (longer one)
[19:14] <stakkars> arigo?
[19:14] <arigo> yes?
[19:15] <stakkars> I'm working on the ovfcheck thing, in targetpypy1
[19:15] <stakkars> the current set of specialized operations doesn't use multiplication at all
[19:15] <stakkars> (which results in funny code, converting forth and back), but that's not the point.
[19:16] <stakkars> I expected some operation, followed by the ovfcheck call, but what -6*-7 generates is different:
[19:16] <stakkars> in
[19:17] <stakkars> fn_mul__Int_Int
[19:17] <stakkars> block2:
[19:17] <stakkars> v6 = CONV_TO_OBJ_int(v5); if (PyErr_Occurred()) FAIL(err2_2)
[19:17] <stakkars> OP_SIMPLE_CALL((gskippedfunc_ovfcheck, v6, NULL), v7, err2_3)
[19:18] <stakkars> I wonder why this is generated at all?
[19:18] <arigo> it's a regular call as you expected,
[19:19] <arigo> but the ovfcheck() function itself is skipped because genc realizes that the annotator didn't annotate it at all (actually it didn't drag it in the flowgraphs dict)
[19:19] <stakkars> sure, I talked with Samuele about how to do the special casing for this.
[19:19] <stakkars> but what I would understand if the check is generated after some operation like a+b
[19:20] <stakkars> here, we have conversion to obj, which cannot overflow, and other errors are handled in the first line, already.
[19:20] <stakkars> how comes?
[19:21] <arigo> because nothing was done for ovfcheck() in genc,
[19:21] <pedronis> stakkars: the ovfcheck transformation
[19:21] <pedronis> needs to be done before the typer put is stull in
[19:21] <pedronis> stuff in
[19:21] <pedronis> likely
[19:21] <arigo> so it just says, "oh, an add() annotated with SomeIntegers, let's do it as a plain C add"
[19:22] <stakkars> I understand, but I don't understand what'sgoing on in the code that I posted.
[19:22] <arigo> it is calling a dummy version of the the Python function ovfcheck() as a Python function
[19:22] <pedronis> the code is calling the fallback version of ovfcheck
[19:23] <pedronis> because ovfcheck was skipped
[19:23] <pedronis> and the fallback takes PyObjects
[19:23] <pedronis> so the conversion
[19:24] <stakkars> arghh, so this conversion is just for the pyobject ovfcheck, which is not called at all.
[19:24] <pedronis> which will just throw a I was skipped
[19:27] <stakkars> ok, but in block 1 , we already have
[19:27] <stakkars> block1:
[19:27] <stakkars> v4 = CONV_FROM_OBJ_int(v3); if (PyErr_Occurred()) FAIL(err1_2)
[19:28] <stakkars> which means to me that block 2 is obsolete
[19:30] <arigo> stakkars: there are too many useless conversions all around for now
[19:30] <stakkars> fine with me, I just try to figure out what I want to match and replace :-)
[19:30] <arigo> stakkars: they are just a measure of how much progress we still have to do
[19:31] <arigo> stakkars: in addition, gskippedfunc_ doesn't mean "skip this call" but rather "complain about calling a function that the annotator says should not be there"
[19:31] <pedronis> indeed
[19:31] <stakkars> sure, this is meant as an open end.
[19:31] <pedronis> stakkars: you really need to match and replace
[19:32] <pedronis> before you get to this
[19:32] <stakkars> yes, I was searching for something helpful, but the generated code is probably too late.
[19:32] <arigo> the typer.py and ctyper.py need to be made more capable to match and replace this pair of operations: add followed by simple_call(ovfcheck)
[19:32] <stakkars> yes, but here is no add!
[19:33] <arigo> uh?
[19:33] <arigo> mul in this case
[19:33] <arigo> you should really be looking at the flow graph, not the generated code
[19:33] <stakkars> that's what I'm telling all the time. We have a multiplicaiton here, and I didn't expect this at all
[19:33] <stakkars> -6 * -7
[19:34] <arigo> so you're looking at mul__Int_Int
[19:34] <pedronis> you don't expect a multiplication in mul__Int_Int?
[19:35] <stakkars> arigo:yes.
[19:36] <stakkars> pedronis: no I don't expect the overflow check problem, because mutiplication is not defined in int_include.h
[19:36] <arigo> the source code contains ovfcheck(x*y) right?
[19:36] <stakkars> didn't look at the source at all :- ) ...
[19:36] <arigo> well look either at the source or at the flow graph, and you'll see a mul operation followed by simple_call(ovfcheck)
[19:37] <arigo> this pattern of two operations should be detected and replaced
[19:37] <stakkars> ah, ok (yes, I actually just wondered becuase it looked so different. Right, I should avoid the generated code)
[19:38] <pedronis> yes, patching the generated code makes no sense
[19:38] <arigo> also look before the ctyper's specialization, which modifies the flow graph.
[19:38] <pedronis> that means -no-t
[19:38] <stakkars> pedronis: you did not assume that I would do that, did you?
[19:39] <pedronis> no
[19:39] <pedronis> I was just wondering why you worried about the code as it look now
[19:39] <stakkars> because I naively assumed I would find an operation, followed by the call to ovfcheck, which was an error of mine.
[19:40] <pedronis> that what you find before ctyper does it thing
[19:40] <pedronis> its thing
[19:40] <stakkars> and I misunderstodd, because I didn't find a mul_int_int operation in the int_include.h file.
[19:41] <pedronis> well, you need either need to teach typer to look for pattern at end of a block
[19:41] <pedronis> and probably before munging its content
[19:42] <pedronis> or a transformation done after annotation but before typer
[19:42] <stakkars> I thought to attach things to transform.py
[19:42] <stakkars> and addsomething that searches this and generates different instructions.
[19:42] <arigo> stakkars: in this case it makes some sense, as the transformation is probably interesting for any back-end
[19:43] <pedronis> arigo: yes, that is what I was thinking too
[19:43] <pedronis> stakkars: yes, a.simplify is done before a.specialize
[19:43] <stakkars> yes, I guess that any regular backend will needto do this, so that's why I looked there.
[19:44] <pedronis> but now without -no-t the translate_pypy.py will show the graph after specialization
[19:44] <stakkars> and we need to spit out certain int int operations with and without built-in checks.
[19:45] <arigo> stakkars: I guess replacing the two operations with a single "mul_ovf" operation should be enough
[19:45] <pedronis> yes
[19:46] <pedronis> the simple_call to ovfcheck will appear at end of a block
[19:46] <stakkars> yes. completely clear to me, I was just not understanding what the code was trying to do.
[19:46] <pedronis> and the op before that
[19:46] <arigo> it is an independent issue to then teach ctyper.py to replace mul_ovf with mul_ovf_int_int when the operation is done on ints, and finally to implement mul_ovf_int_int in int_include.h
[19:46] <stakkars> sounds reasonable
[19:48] <stakkars> other question: what is the essential difference between targetpypy0 and 1
[19:49] <stakkars> does it avoid creating to much stuff, or is there more to it?
[19:50] <pedronis> targetpypy1 is more similar to targetpypy
[19:50] <pedronis> it picks a subset of the std object space
[19:50] <pedronis> but
[19:50] <stakkars> ok, I just can't remember the reasoning
[19:51] <pedronis> while targetpypy takes interpreter+all of the std object space
[19:51] <stakkars> then I mean these two,of course
[19:51] <pedronis> targetpypy0 uses a dummy object space
[19:51] <pedronis> that I wrote
[19:51] <pedronis> and was useful trying to annotate basically just the interpreter
[19:51] <pedronis> and get some progress
[19:52] <stakkars> I see. and targetpypy1 does use less that targetpypy, because it explicitly touches the multi methods it needs?
[19:52] <pedronis> yes, it basically extract the core impl of the mul multimethod
[19:53] <pedronis> so that not all of the space and interpreter are brought in
[19:53] <pedronis> so it touches less code than both targetpypy and targetpypy0
[19:53] <stakkars> is this still necessary to do? I thought the annotator is so smart now that it just finds out what to skip?
[19:54] <stakkars> (sorry, I can try this out myself instead of bogging you)
[19:55] <pedronis> no, targetpypy is basically targetpypy1 without the extracting
[19:55] <pedronis> and it brings everythin in
[19:55] <pedronis> because descroperation gets involved
[19:55] <stakkars> aha
[19:56] <pedronis> you get to touch lookup etc
[19:56] <pedronis> and ad get_and_call etc
[19:56] <pedronis> and the transitive closure results to be all
[19:57] <stakkars> so we don't figure out that we have a constant, here.
[19:58] <stakkars> I mean, the descroperation might be done at compile time
[19:59] <stakkars> (sorry if I'm talking nonsense, I assumed that the mul would be found by a descroperation)
[19:59] <arigo> the annotator doesn't really do constant propagation so far.
[19:59] <arigo> only the flow object space does, so it's limited to intraprocedural propagation
[19:59] <stakkars> so if it did,, it would not pull the interpreter in at all
[19:59] <arigo> probably not, indeed
[20:00] <stakkars> further, it would compute -6*-7 and just produce the result:-)
[20:00] <arigo> :-)
[20:00] <arigo> at some point, we need a real entry point that basically starts a py.py command-line interpreter
[20:01] <stakkars> yes. I also expect that rpystone would get compiled away, completely, unless we have the command line argument of how many passes to do
[20:01] <stakkars> well, even that might not help, unless we use the results.
[20:02] <pedronis> stakkars: the flowspace doest not constant-fold calls
[20:02] <stakkars> but it would be great if something would do this, soon.
[20:03] <stakkars> anyway, I now got a full understanding of these hows and whys, many thanks!
[20:03] <pedronis> well, it could
[20:03] <stakkars> it could try to evaluate a call and see if it constant
[20:03] <pedronis> but is not simple to know whether the call depens on things that are not constant
[20:03] <pedronis> so it's a risky change
[20:04] <pedronis> and is not that useful for our purposes right now
[20:04] <arigo> yes, I'd rather let the annotator do the constant propagation itself
[20:05] <arigo> it's a bit of a duplication, but we should be able to easily factor out the necessary logic
[20:05] <stakkars> you mean instead of the flow space
[20:05] <arigo> or in addition
[20:05] <arigo> some constant propagation is very useful in the flow space
[20:05] <arigo> but anyway it's not needed at the moment.
[20:05] <pedronis> yes
[20:06] <pedronis> I don't think that enough constant-folding involving calls will be our major problem
[20:06] <pedronis> that not
[20:07] <stakkars> well, it would have the nice side-effect that we don't need the different targets any longer
[20:07] <pedronis> well, we would need more carefully crafted targets
[20:08] <pedronis> to avoid folding we don't want
[20:08] <stakkars> yes, but that's much nicer than the opposite
[20:08] <pedronis> I'm not sure
[20:09] <pedronis> anyway it's pretty theoretical
[20:09] <arigo> stakkars: regard the current targets as hacks to get increasingly complex code bases, for testing purpose
[20:11] <stakkars> sure. I just didn't see why eliminating constant stuff would be just great
[20:11] <stakkars> focusing on targetpypy1
[20:11] <pedronis> well, but with complete folding targetpypy1 would just give a constant
[20:12] <stakkars> yes, that's great to me.
[20:12] <pedronis> but it would pretty unuseful as a target
[20:12] <stakkars> if you want some code, you need to provide data which is not predictable.
[20:13] <stakkars> so I'd use an expression that computes something from time.time or use some cmd arg
[20:13] <arigo> that's theoretically a very nice idea, but at the moment it's not a priority
[20:13] <stakkars> the reason I like this is that this is into the direction of building C extensions.
[20:14] <stakkars> ok, fijne.
[20:14] <arigo> introducing it right now has the disadvantage of break a few things like targetpypy1.
[20:14] <pedronis> yes
[20:14] <pedronis> and basically you would need very generic targets like exec
[20:15] <pedronis> to touch significant part of the codebase
[20:15] <pedronis> most other kind of code would fold a lot away
[20:16] <stakkars> just to see if I understand something:
[20:16] <stakkars> if I know a and b are immutable for instance, and their type==int is immutable, to, then I may
[20:17] <stakkars> evaluate the descriptor of the add at compile time
[20:17] <stakkars> and evaluate the operation at compile time, too.
[20:19] <arigo> that's pretty vague, but that's something the annotator could do
[20:20] <pedronis> stakkars: you mean int or our int implementation
[20:20] <stakkars> I meant if the things involved are all constants, it is safe to evaluate
[20:21] <pedronis> but the flowspace does add constant ints
[20:22] <stakkars> yes, that's different.
[20:23] <pedronis> sorry but type int and immutable means constant int
[20:23] <pedronis> unless you mean our int implementation
[20:23] <stakkars> I'm talking of what the targets are doing, of course.
[20:24] fredrik (fredrik@c83-248-135-181.bredband.comhem.se) joined #pypy.
[20:24] <pedronis> then you mean W_IntObject
[20:26] <stakkars> yes. You can find out that these are constant, too.
[20:26] <pedronis> one problem with folding call is that you need to whether they have side-effects or not
[20:27] <stakkars> ah, folding is ok on std objspace, but not on trace obj space for instance.
[20:27] <arigo> stakkars: you are looking at things at the wrong level
[20:27] <pedronis> the flow space just sees a call, it is no idea what is about
[20:28] <pedronis> it has
[20:28] <arigo> stakkars: it is confusing to think that it is PyPy we are translating, let's think about examples like rpystone
[20:29] <stakkars> you mean I was one level off? Maybe, and I'm not sure if +1 or -1 :-)
[20:30] <arigo> rephrase your remarks with rpystone -- no more stdobjspace anywhere
[20:31] <stakkars> ok, then it depend on what a side effect is. The global objects are modified but never read, again.
[20:32] <pedronis> but you don't know because some global instances can be mutable
[20:32] <pedronis> or in general you could have a print
[20:32] <pedronis> so you would really need to analyze what the function does
[20:32] <pedronis> before knowing whether folding is safe
[20:33] <stakkars> I would probably not analyse, but do some tagging of the basic blocks, the rest can be deduced.
[20:33] <pedronis> it is not a low-hanging fruit at all
[20:34] <stakkars> "a thing is foldable if all involved parts are foldable or foldable primitives"
[20:34] Action: stakkars thinks he should stop wasting time and implement nice overflows
[20:35] <arigo> stakkars: yes :-) note that an easier intermediate step is the constant-folding that the annotator could do, which doesn't mean inlining calls, but creating functions that are basically empty because they will end up manipulating constants only.
[20:35] <arigo> stakkars: but even that should be postponed now
[20:40] <stakkars> I like this idea! Not folding at all, but having functions that collapse to nothing...
[20:40] <stakkars> s/folding/inlining/
[20:40] <arigo> that's very much what Psyco does :-)
[20:42] <pedronis> btwwith foldindm I did not mean inlining
[20:42] <stakkars> I knew that you trapped us all into this project, just to get your Mega-Psyco done :-)
[20:43] <pedronis> well, inlining is something we may need if don't get enough of it for free by the compilers
[20:44] <stakkars> well, is there so much difference? between folding and collapsing I mean.
[20:44] <stakkars> inlining is in fact a different beast, and will be needed,too. See md5.py
[20:45] <stakkars> ick, I'm coding :-)
[20:45] <pedronis> with foulding I meant constant-folding
[20:45] <pedronis> that mean f(5,4) -> constant result
[20:46] <stakkars> that means to do some evaluation of f
[20:46] <stakkars> so fmust not have a side effect.
[20:46] <pedronis> yes
[20:47] <pedronis> or do the propagating upward in the annotator of "I just produce constants and don't have side-effects"
[20:49] <stakkars> a bit more: we would need to specialize on the constant args case
[20:49] <pedronis> yes, but I don't see much of that as usufual at compile-time
[20:49] <pedronis> of course moved at runtime is a different thing
[20:53] <stakkars> I meant the annotator could not do your "..." without specializing, because "mul" for instance is not producing constants in general.
[20:56] <pedronis> my point is tha apart toy examples is not completely typical to have calls in a program that can be computed at compile-time apart cases where the programmer left them there
[20:57] <pedronis> so specializing is a more useful as runtime activity
[20:59] <arigo> hum, what is the type of the C expression
[20:59] <arigo> p->z
[20:59] <arigo> given struct { int z[]; } *p
[21:00] <arigo> I think it is "int*"
[21:00] <arigo> but then I don't understand why the compiler doesn't complain on
[21:00] <arigo> &(p->z)
[21:00] <arigo> ?
[21:04] <stakkars> it is just p
[21:04] <pedronis> arigo: I get a warnign with &
[21:04] <arigo> pedronis: ah
[21:04] <arigo> stakkars: p->z also has the same value as p
[21:05] <stakkars> ough
[21:06] <stakkars> pedronis: what is the warning? number of derefs?
[21:07] <arigo> I'd better not rely on it then and make two C-level graph operation to get a pointer to a field of a struct, depending on whether it is an array or not
[21:08] <pedronis> arigo: yes
[21:08] <pedronis> stakkars: incompatible pointer type
[21:13] <stakkars> fine
[21:14] <pedronis> arigo: the type of &z is int** it seems
[21:14] <arigo> pedronis: I can't do int** q = &p->z; without a warning, though
[21:14] <stakkars> physically it's p, but different type
[21:15] <arigo> I can do void* q = &p->z; but I can't seem to type q with anything else
[21:15] <arigo> the resulting pointer is indeed equal to p
[21:15] <pedronis> yes
[21:16] <arigo> but I doubt it's an int**, because (int**)p is meaningless.
[21:16] <pedronis> I was trying
[21:16] <arigo> :-)
[21:16] <pedronis> with both int z[] and it insided a struct
[21:16] <stakkars> hmm, the pattern for the ovfcheck is a bit different.
[21:16] <pedronis> for an array outside a struct int** is the type
[21:17] <arigo> ah?
[21:17] <arigo> hpk: svn down?
[21:18] <stakkars> mul__int_int does
[21:18] <stakkars> try:
[21:18] <stakkars> z = ovfcheck(x * y)
[21:18] <stakkars> except OverflowError:
[21:18] <stakkars> etc
[21:19] <stakkars> this creates two blocks instead of the one I expected.
[21:19] <arigo> warning, recoversvn :-)
[21:19] <stakkars> one is getattr, getattr, mul - exitswitch
[21:19] <arigo> stakkars: oups oups oups, it's catching the implicit float overflow of the '*'
[21:20] <pedronis> yes
[21:20] <stakkars> other is simple_call(function ovfcheck) -> exitswitch
[21:20] <stakkars> which is correct from a primitive POV, since the ovfchek *and* the operation are elcosed by the try block ;-)
[21:21] <stakkars> yes
[21:21] <stakkars> and this also lets me understand the C code better
[21:22] <stakkars> yes, it wants to do failedto implement
[21:23] Action: pedronis is going home to eat
[21:23] <stakkars> both tryto branch to this same exception
[21:24] <arigo> it's messy
[21:24] <stakkars> so I guess this is not the situation that I want to match, but a bug?
[21:24] <stakkars> or should I simplysay this is what I need to fix, and merge the blocks into one checked operation
[21:24] <arigo> that's fragile
[21:25] <pedronis> well, unless the flow graph knows about ovfcheck it is hard to fix
[21:25] <stakkars> are there any cases that would produce a different pattern?
[21:25] <arigo> a clean but not nice solution would be to use a different exception than OverflowError
[21:26] <stakkars> in which case
[21:26] <arigo> stakkars: yes, a mere ovfcheck(x*y) with no handler (because it's in the parent)
[21:26] <arigo> for ovfcheck()
[21:27] <arigo> if ovfcheck() raises another exception and we catch it instead of OverflowError, we don't have this ambiguity
[21:27] <stakkars> this would have the handler just for the operation
[21:27] <arigo> but maybe it's not needed
[21:27] <arigo> yes
[21:27] <stakkars> well, this isn't so bad!
[21:27] <stakkars> the fact that we have two block is just because there are the handlers.
[21:28] <stakkars> and the generalpattern is, that we have two operations that can be collapsed into one,
[21:28] <arigo> I guess so -- it can be solved with patterns that are hackish enough :-)
[21:28] <stakkars> because a) x+y followed by ovfcheck is a known pair, and b) the excpetion handlers are the same.
[21:29] <pedronis> bye
[21:29] pedronis (pedronis@ratthing-b246.strakt.com) left irc: "Chatzilla 0.9.67 [Firefox 1.0.2/20050325]"
[21:29] <stakkars> bye
[21:29] <arigo> or a two-step approach would do,
[21:29] <arigo> like figuring out first that the exc handler for the mul() isn't needed
[21:30] <stakkars> if ovfcheck raises another exception, the problem doesn't change somuch
[21:30] <arigo> because it's an operation with integers
[21:30] <stakkars> ok, and by existence of the ovfcheck, we recognize that we do wat it, again
[21:30] <stakkars> want it
[21:30] <arigo> yes
[21:31] <stakkars> how comes that the exc handler for mul is generated?
[21:31] <arigo> so actually we can ignore the first handler completely because of the types
[21:31] <arigo> because floating point mul still raises OverflowError
[21:31] <arigo> so this first handler is only for the case where the arguments are floats
[21:31] <stakkars> but we have int :-|
[21:31] <arigo> yes
[21:31] <arigo> the flow graph doesn't know that
[21:32] <arigo> and the annotator isn't subtle enough to avoid following some exception handlers when they are not needed
[21:32] <arigo> so precisely, I suggested a first phase where such handlers are detected and removed manually by transform.py
[21:32] <stakkars> hmm! Looks like theywould better work a bi closer together?
[21:32] <stakkars> bit
[21:32] <arigo> well, not necessarily
[21:33] <stakkars> ok, one washing process that picks such dirt
[21:33] <arigo> in this case, if you want a nice and clean solution, only the annotator should be told about operations that don't raise based on types
[21:33] <stakkars> then maybe re-merge blocks,
[21:33] <stakkars> then modify the operations.
[21:33] <arigo> if the annotator is told not to follow the exception handler, then an existing function in transform.py will drop it.
[21:34] <stakkars> that means I have to stick something in the special cases I guess
[21:34] <stakkars> no this was bad
[21:35] <arigo> "specialcase" is for the flow only
[21:35] <arigo> stakkars: the procedure you describe is fine, what I am wondering about is if we could replace the "washing dirt" point with annotator knowledge.
[21:35] <stakkars> ann_override or something
[21:36] <stakkars> keeping the annotator to produce the dirt at all
[21:36] <arigo> not so easy, ann_override is for built-in functions
[21:36] <arigo> look at annrpython.py, def flowin(self):
[21:36] <stakkars> yes, I'm a newbie with these, but just hting I said the right thing
[21:36] <stakkars> think
[21:37] <arigo> flowin() has two parts: a for loop over block.operations enclosed in a huge try:except:
[21:37] <arigo> and a for loop over exits
[21:38] <stakkars> brrbrrbrrbrr --- Yes
[21:38] <arigo> exits is normally block.exits, i.e. all the exits of the block,
[21:38] <arigo> but in some cases it is only some exits.
[21:38] <arigo> in this case the annotator only follows the given exits.
[21:38] <arigo> any exit never followed will be cut off by transform.py later
[21:39] <stakkars> you mean I don't give ti the excpetion exists, but something else?
[21:39] <arigo> ?
[21:39] <arigo> ah yes
[21:39] <arigo> you only give it the None (non-exceptional) exit
[21:39] <stakkars> I drop those of the exception
[21:39] <arigo> if the last operation is 'mul', and if it returned a SomeInteger.
[21:40] <stakkars> do we have more cases like these?
[21:40] <arigo> possibly
[21:41] <stakkars> so I need something to register.
[21:41] <arigo> in theory, we should go over the list of implicit_exceptions in objspace.py
[21:41] <arigo> and register somewhere that with some types they cannot really occur
[21:41] <stakkars> well, and build ---exactly that!
[21:41] <arigo> like KeyError for a getitem with a list.
[21:42] <arigo> a last note, block.exitswitch == Constant(last_exception) when it's an exception handler,
[21:42] <arigo> and the non-exceptional exit is always the first one in block.exits
[21:44] <arigo> when in doubt, the reference for the flow graph model is checkgraph() in objspace/flow/model.py,
[21:44] <arigo> which should accept all valid graphs and only these.
[21:47] <stakkars> maybe we could put such negative lists of implicit exceptions into the types, themselves?
[21:49] <arigo> ok if you store them from outside, using class __extend__(SomeInteger): ...
[21:49] <arigo> as is done e.g. in unary.py
[21:49] <arigo> to keep the classes themselves as compact as possible
[21:50] <stakkars> why is this better? ahh
[21:50] <stakkars> then itmakes more sense tohave a compat table where I doallof it. ?
[21:50] <stakkars> compact
[21:50] <arigo> yes, probably
[21:51] <stakkars> because we are looking somewhere different, anyway. Just a re-lookup for the specific type,maybe.
[21:51] <arigo> the nicer would be a way to say it directly inside def add() for integers, but it would be messy
[21:51] <stakkars> anyway it seems to be a typical symptom that whatever simplething I'm trying, it turns into a problem :-)
[21:52] <arigo> :-)
[21:52] <stakkars> well, it dependson how often we have this.
[21:52] <arigo> hey, a nice enough way:
[21:52] <arigo> as an attribute of the add() method
[21:52] <stakkars> yes, I thought of it!
[21:52] <arigo> meaning "I promize I don't raise the following exception" :-)
[21:52] <arigo> or "only the following exceptions", more probably
[21:53] <stakkars> but that again goes into the type's implementation
[21:53] <stakkars> or what was it you said should be as tight as possible?
[21:53] <arigo> that's ok, I wanted to say that the classes of model.py should be as tight as possible
[21:53] <arigo> sorry, I missed your point then
[21:54] <stakkars> so I would add it as an attribute to the implementing function, and that's it.
[21:54] <arigo> makes sense
[21:55] <stakkars> on the other hand, some centralpoint might be nice, too. Well, I could do it from elsewhere, with the side effect ofpuloling certain files in, early
[21:55] <stakkars> of pulling
[21:55] <arigo> I think the distributed version makes sense
[21:56] <arigo> it's really like the 'throws' that Java programmers put after a function header
[21:56] <stakkars> alternatively, we are already creating the whole table in one shot. Maybe providing an exception type list would do as well?
[21:56] <arigo> I vote for 'throws' :-)
[21:57] <stakkars> add__int_int_.__throws__ = blah
[21:57] <arigo> that's also clearer in the sense that no 'throws' attribute means 'I don't know' and not 'cannot throw anything'
[21:58] <arigo> in binaryop it's more (SomeInteger, SomeInteger):... add.throws = []
[21:58] <arigo> but that's the idea :-)
[21:58] <stakkars> ah, sure, it doesn'tgo to int, it goes to binaryop
[21:59] <stakkars> so that's for my homework,and I go now?
[21:59] <arigo> intobject.py? doesn't exist. who ever heared about intobject.py? no such thing! :-)
[21:59] <arigo> ok, see you later :-)
[22:00] <stakkars> oops, missed it vanishing. moooving tagrets all around
[22:00] <stakkars> ciao
[22:00] <arigo> :-)
[22:01] arigo (~arigo@vicky.ecs.soton.ac.uk) left irc: "dinner"
[22:12] pedronis (~Samuele_P@c-3c8b70d5.022-54-67626719.cust.bredbandsbolaget.se) joined #pypy.
[22:21] <hpk> pedronis: i sent you mail regarding pypy-eu-svn ...
[22:23] <pedronis> hpk: got it
[22:29] <hpk> pedronis: you should have got two commit mails now
[22:29] <pedronis> hpk: yes
[22:29] <hpk> one problem
[22:30] <hpk> commits to extradoc now go both to pypy-eu-svn and pypy-svn
[22:30] <hpk> i am unsure what to do with that
[22:31] <hpk> i tend to just leave it and live with it
[22:33] <pedronis> I think I can live with the double mails
[22:33] <hpk> extradoc is not used that much, probably, anyway and we can change it later
[22:34] Action: hpk has the feeling that eu partners will tend to put everything in eu-tracking ...
[22:47] stakkars (~tismer@rosine174.inf.fu-berlin.de) left irc: Read error: 113 (No route to host)
[22:48] <hpk> pedronis: ok, i subscribed everyone ...
[22:48] <hpk> (little accident, i originally wanted to invite people, but well)
[22:48] <hpk> i had my cat working to and fro before me when sending out the mail :-)
[22:59] <hpk> pedronis: are you available for a bit of dumb help regarding the tests?
[23:00] <hpk> (i.e. it doesn't require much active brain cells, i guess)
----- silence for 25 minutes ----- [23:25] <pedronis> hpk: oops, hpk, I was distracted by other windows
[23:25] <hpk> no problem, it went quicker then i thought
[23:25] <hpk> i am trying to get all tests to have a sensible test type
[23:26] <hpk> but as cfbolz said earlier today, probably all currently 'UnknownTestModules' are simply 'RunningAppModules'
[23:26] <hpk> i will do an according commit soon
[23:26] <pedronis> apart doctests unless they are all going to unittests
[23:26] <pedronis> s/to/through
[23:26] <hpk> and then i implenent an '--resultdir=XXX' option which writes out reports accordingly
[23:27] <hpk> pedronis: yes, sure, the list will still need to get refined
[23:27] <hpk> when we have doctest support, it's still all slightly flaky though
[23:27] <hpk> because i am not sure that i have patched all invocations of doctest/unittest appropriately
[23:28] <pedronis> patch invocations?
[23:28] <hpk> py.test substitutes invocations to run_unittest() and run_doctests()
[23:28] <hpk> rather intercepts those invocations to do its collection process
[23:28] <pedronis> ah, yes
[23:29] <hpk> (the one you see with 'py.test --collectonly')
[23:29] <pedronis> it seems thare are not that many doctest tests
[23:29] <hpk> they got more in python2.4 and 2.5 i think
[23:30] <hpk> i am going to tackle this in the next days
[23:30] <hpk> but first the --resultdir option
[23:32] <hpk> pedronis: i have documented the test session/collection mechanism somewht more, btw
[23:32] <hpk> http://codespeak.net/py/current/doc/test.html#collecting-and-running-tests-implementation-remarks
[23:33] <hpk> in case you are interested to skim it
[23:43] _hannes (ppzosq@82.140.29.171) left irc: "utz utz utz"
[00:00] --- Wed Apr 27 2005