[00:06] aleale (~chatzilla@0x535ca5a5.hrnxx9.adsl-dhcp.tele.dk) left irc: "Chatzilla 0.9.68a [Firefox 1.0.3/20050414]"
[00:09] fredrik (fredrik@c83-248-135-181.bredband.comhem.se) left irc: "http://fredrikj.net";
[00:14] yuuh (gtrivq@82.140.29.224) left irc: "utz utz utz"
----- silence for 23 minutes ----- [00:37] stakkars (~tismer@82.140.29.224) joined #pypy.
[00:39] <stakkars> targetrpystone is working!
[00:40] <stakkars> before compilation: 35175.9 pystones/second
[00:40] <stakkars> fater compilation: 92963.7 pystones/second
----- silence for 4 hr and 25 minutes ----- [05:05] dialtone (~dialtone@host118-0.pool21345.interbusiness.it) left irc: "This computer has gone to sleep"
----- silence for 19 minutes ----- [05:24] stakkars (~tismer@82.140.29.224) left irc: Read error: 113 (No route to host)
----- silence for 1 hr and 40 minutes ----- [07:04] idnar (mithrandi@idnar.user) left irc: Read error: 60 (Operation timed out)
----- silence for 54 minutes ----- [07:58] dialtone (~dialtone@host118-0.pool21345.interbusiness.it) joined #pypy.
[07:58] dialtone (~dialtone@host118-0.pool21345.interbusiness.it) left irc: Remote closed the connection
----- silence for 40 minutes ----- [08:38] arre (ac@ratthing-b3fa.strakt.com) joined #pypy.
[08:49] dialtone (~dialtone@host158-206.pool8252.interbusiness.it) joined #pypy.
----- silence for 50 minutes ----- [09:39] dialtone (~dialtone@host158-206.pool8252.interbusiness.it) left irc: "This computer has gone to sleep"
[09:48] ludal (~ludal@logilab.net2.nerim.net) joined #pypy.
----- silence for 1 hr and 8 minutes ----- [10:56] aleale (~chatzilla@0x535ca5a5.hrnxx9.adsl-dhcp.tele.dk) joined #pypy.
----- silence for 29 minutes ----- [11:25] pedronis (pedronis@ratthing-b246.strakt.com) joined #pypy.
----- silence for 41 minutes ----- [12:06] idnar (mithy@idnar.user) joined #pypy.
----- silence for 1 hr and 45 minutes ----- [13:51] arigo (~arigo@ratthing-b401.strakt.com) joined #pypy.
----- silence for 46 minutes ----- [14:37] arre (ac@ratthing-b3fa.strakt.com) got netsplit.
[14:37] idnar (mithy@idnar.user) got netsplit.
[14:37] arigo (~arigo@ratthing-b401.strakt.com) got netsplit.
[14:37] pedronis (pedronis@ratthing-b246.strakt.com) got netsplit.
[14:37] ludal (~ludal@logilab.net2.nerim.net) got netsplit.
[14:37] hpk (~hpk@merlinux.de) got netsplit.
[14:37] etrepum (bob@ayunami.redivi.com) got netsplit.
[14:37] tic (~vision@s-213-115-110-186.sme.bredbandsbolaget.se) got netsplit.
[14:38] arigo (~arigo@ratthing-b401.strakt.com) returned to #pypy.
[14:38] idnar (mithy@idnar.user) returned to #pypy.
[14:38] pedronis (pedronis@ratthing-b246.strakt.com) returned to #pypy.
[14:38] ludal (~ludal@logilab.net2.nerim.net) returned to #pypy.
[14:38] arre (ac@ratthing-b3fa.strakt.com) returned to #pypy.
[14:38] hpk (~hpk@merlinux.de) returned to #pypy.
[14:38] etrepum (bob@ayunami.redivi.com) returned to #pypy.
[14:38] tic (~vision@s-213-115-110-186.sme.bredbandsbolaget.se) returned to #pypy.
----- silence for 22 minutes ----- [15:00] <hpk> arigo: hi
[15:01] <hpk> arigo, pedronis: i have better keyword support ready now
[15:01] <hpk> which i am about to commit
[15:01] <hpk> the 'core' and 'noncore' options are basically gone
[15:02] <hpk> you now say e.g.: "py.test -k core"
[15:02] <hpk> or: py.test -k "core -test_mutants"
[15:02] <hpk> it's basically google-syntax right now
[15:04] <arigo> nice!
[15:04] <hpk> let me commit then i explain further (there is more :-)
[15:05] Action: hpk commited
[15:05] <hpk> there are also articifical keywords like: error, ok, timeout
[15:05] stakkars (~tismer@rosine101.inf.fu-berlin.de) joined #pypy.
[15:05] <hpk> arigo: so you can say now: py.test -k "core error"
[15:05] <stakkars> hi
[15:06] <arigo> hpk: our user@host/test_xyz.out files should be removed, right?
[15:06] <arigo> hi stakkars
[15:06] <hpk> arigo: that's in conftest.py:778
[15:06] <hpk> arigo: yes
[15:06] <hpk> stakkars: hi!
[15:06] <stakkars> I have rpystone running :-)
[15:07] <hpk> and how fast is it?
[15:08] <hpk> (not that it matters much at this point, but i am still curious)
[15:08] <stakkars> right now a factor of 3-3.5
[15:09] <stakkars> but I'm not as keen on the speed, too, but that it works at all!
[15:10] thingie25 (~rmt38@valhalla.ccp.cc) left irc: Read error: 104 (Connection reset by peer)
[15:10] <hpk> yes, that's nice
[15:15] thingie24 (~rmt38@valhalla.ccp.cc) joined #pypy.
[15:21] <hpk> arigo: somebody released something called "pytest" :-( (on python-announce)
[15:21] <arigo> ouack
[15:22] <hpk> i wrote him a mail, telling about the name colision, let's see
[15:25] arre (ac@ratthing-b3fa.strakt.com) left irc: "using sirc version 2.211+KSIRC/1.3.11"
----- silence for 20 minutes ----- [15:45] <hpk> arigo: i am talking to Chad on #pylib already :-)
[15:47] yuuh (wkfehf@82.140.29.224) joined #pypy.
[15:52] dialtone (~dialtone@host158-206.pool8252.interbusiness.it) joined #pypy.
[15:56] idnar (mithy@idnar.user) left irc: "kbye"
[16:03] dialtone (~dialtone@host158-206.pool8252.interbusiness.it) left irc: "This computer has gone to sleep"
[16:15] <hpk> arigo: resolved, see http://www.zetadev.com/software/pytest/
[16:17] <arigo> I see
[16:17] <hpk> arigo: he is actually doing his test thingie by interpreting the AST
[16:18] <arigo> yes, I was wondering
[16:23] <hpk> hum, some tests fail if one is not in 2.3.4/test
[16:23] <hpk> double-hum
[16:24] <hpk> test_cpickle does "from pickletester import ..."
[16:24] <hpk> but pickletester.py is in modified
[16:24] <hpk> while test_cpickle is unmodified
[16:25] <hpk> so i guess test_cpickle should be copied to modified as well
[16:26] ludal (~ludal@logilab.net2.nerim.net) left irc: "Download Gaim: http://gaim.sourceforge.net/";
[16:29] <arigo> hpk: why?
[16:30] <arigo> I see the import fails but I have no clue why
[16:30] <hpk> 2.3.4/test/test_cpickle.py does "from pickletester ..."
[16:30] <hpk> and this is run as main, has no __path__
[16:31] <hpk> so it can't find the modified pickletester (note the missing 'test.' prefix)
[16:31] <hpk> the import in the result web page is failing for a different reason
[16:31] <arigo> how does this work on top of CPython?
[16:31] <hpk> because i routinely run the tests from lib-python
[16:31] <hpk> you have to be in the test directory to run the regrtests i guess
[16:32] <hpk> so it's found from the current directory i guess
[16:32] <arigo> no, it works on CPython...
[16:33] <arigo> different sys.path magic, maybe
[16:33] <arigo> if so, it's really a bug of py.py
[16:33] <arigo> I'm precisely cleaning up py.py and main.py a bit
[16:34] <hpk> yes, then CPython apparently puts dirname(pythonfile) into sys.path
[16:34] <hpk> indeed, it does
[16:39] <hpk> arigo: note, however, that the above test_cpickle problem will remain
[16:41] <arigo> hpk: right
[16:43] <hpk> so i am copying it over
[16:43] <hpk> (although the original test passes, btw, but takes 10000 seconds or something)
[16:45] <arigo> hpk: ok, but in addition it might be nice to fix test_cpickle to do an absolute import instead
[16:45] <hpk> sure
[16:45] <arigo> but we need to move it in order to change it, and after it's moved it doesn't need to be changed any more :-)
[16:46] <hpk> yes, it's a reverse chicken-egg situation
[16:46] <arigo> a bootstrapping non-problem :-) good
[16:46] <hpk> (whatever that means :-)
[16:51] Action: hpk wonders where he got the idea that test_cpickle.py passed at some point
[16:51] Action: stakkars remembers that it did
[16:53] <hpk> stakkars: currently it doesn't, see http://codespeak.net/~hpk/pypy-testresult/.cache/test_cpickle.html
[16:53] <stakkars> sure
[16:56] yuuh (wkfehf@82.140.29.224) left irc: Read error: 113 (No route to host)
[16:57] yuuh (dkfynkv@82.140.29.224) joined #pypy.
[17:06] Action: hpk -> lunch
[17:13] <stakkars> arigo: ?
[17:13] <arigo> yes?
[17:13] <stakkars> still working on ovfcheck and all the operators (I can't stop now)
[17:14] <stakkars> problem: what I did so far was to use ovfcheck as an indicator that a special operator must
[17:15] <stakkars> be generated. This was bad, because there are operators which don't have overflow but need to be checked.
[17:15] <stakkars> So I'm cleaning this up and restructure a bit.
[17:15] <stakkars> What I'm not sure about:
[17:16] <stakkars> I want torewrite certain operations to ovf/zero/ whatever checked operations.
[17:16] <stakkars> Those will then be rewritten by ctyperto use the right int macros.
[17:17] <stakkars> *is it correct* to use the fact that an operation defines can_only_throw as an idicatorfor rewriting theop?
[17:17] <arigo> i'm a bit confused but it doesn't look very correct a priori
[17:19] <stakkars> in ctyper, I have now lotsof ops like "int_mod_ovf" and "int_mod_ovf_zer"
[17:19] <stakkars> I want to generate the "mod_ovf" and "mod_ovf_zer" in the correct place. Right now it happens
[17:20] <arigo> wait a bit
[17:20] <stakkars> in transform.py, but this imposes that all generators must understand this.
[17:20] <arigo> "mod_ovf" is an operation tha should be put in the flow graph before ctyper sees it
[17:21] <arigo> because all generators are interested in knowing which version of "mod" they need to write
[17:21] <stakkars> ok, that's fine!
[17:21] <arigo> ctyper's role is only to turn mod_ovf into int_mod_ovf, not to do anything special about overflow checking
[17:21] <stakkars> wonderful,exactly what I hoped for.
[17:21] <arigo> :-)
[17:22] <arigo> but what are you doing with can_only_throw?
[17:22] <stakkars> nothing, yet. I did the rewriting only if ovfcheck was in place. But that is wrong.
[17:23] <stakkars> Now I simply remove ovfcheck completely, and have the OverflowError appended to the can_only_throw things.
[17:23] <stakkars> But now I need a new way to decide whether I rewrite operations :-)
[17:24] <stakkars> one idea was the can_only attribute. But well, should I simplyrewrite every primitive operation that has an exception handler?
[17:24] <arigo> I'm still a bit lost. I think the problem is not related to the annotator
[17:24] <arigo> if so, it's not related to can_only_throw or to transform.py and it should go into simplify.py
[17:26] <stakkars> then simplify would rewrite ops even without annotation.
[17:27] <arigo> the annotator was introduced into this picture to get rid of the extra exception handlers
[17:27] <arigo> I think we need a way that still produces the correct result even without the annotator.
[17:27] <arigo> (correct but with too much stuff left)
[17:28] <stakkars> well, the correct result appears when the ovfcheck calls are active.
[17:28] <stakkars> with annotation, I need to remove these calls and generatespecial operators.
[17:29] <arigo> I'm not sure it's related to annotations
[17:29] <stakkars> and only after annotation, I have realknowledge which exceptions can happen.
[17:29] <pedronis> well, but without annotation we assume everything is a PyObject
[17:30] <pedronis> and that ops or a not erased ovfcheck
[17:30] <pedronis> will raise the right exceptions
[17:30] <stakkars> exactly.
[17:31] <arigo> it's still not necessarily related to the annotator
[17:31] <arigo> if we say "officially" that flow graphs can contain either "add" or "add_ovf" operations,
[17:31] <hpk> arigo: did you know that py_compile.py exists?
[17:32] <arigo> then we could have a back-end which produces a PyNumber_Add() and PyNumber_AddNoCheck() operation, if such a thing existed
[17:32] <arigo> putting these add_ovf into the flow graph is an "early" operation
[17:32] <stakkars> I agree. So you say the rewriting of the ops and removalof ovfcheck is a step before annotation
[17:33] <arigo> something that should be done as quickly as possible after the flow graph is generated
[17:33] <arigo> yes, because somehow it's a "correction": the graph is not correctly representing our intent, we are fixing it
[17:33] <arigo> at least in theory
[17:34] <arigo> hpk: no
[17:35] <stakkars> bu in this early state I don't know the types, that means I have to produce op_ex1_ex2_ex3 operations for every
[17:36] <stakkars> exception that is in place, andlater get rid of them when I know it was an int?
[17:38] arigo (~arigo@ratthing-b401.strakt.com) left irc: Read error: 60 (Operation timed out)
[17:48] pedronis (pedronis@ratthing-b246.strakt.com) left irc: "Chatzilla 0.9.67 [Firefox 1.0.2/20050325]"
[17:48] pedronis (pedronis@ratthing-b246.strakt.com) joined #pypy.
[17:50] arigo (~arigo@ratthing-b407.strakt.com) joined #pypy.
[17:51] <arigo> oups, timed out
[17:51] <arigo> I actually typed a lot of random thoughts, then changed my mind, so it's fine :-)
[17:51] <arigo> do you see me now?
[17:51] <stakkars> yes,now I can see you :-) (from the oups)
[17:52] <arigo> ok
[17:52] <arigo> so after some clearing up of minds...
[17:52] <stakkars> with early name mangling, things might get messy. I'd needto rewrite again, even removing stuff.
[17:52] <stakkars> oh,yes?
[17:52] <arigo> you're doing too much
[17:52] <arigo> 'add' means PyNumber_Add, i.e. overflow checking on floats but not on ints.
[17:53] <arigo> so the basic 'add' already does most checks, apart from int overflow
[17:53] <arigo> we only need 'add' and 'add_ovf', the latter being the same but also doing int overflow.
[17:54] <stakkars> but there aderother things than add.
[17:54] <arigo> no, it's the same for ZeroDivisionError
[17:54] <arigo> I think it's safe to say that 'div' checks everything apart from int overflows
[17:55] <arigo> did you have something in mind?
[17:55] <stakkars> hmm. I broke everything down into the possible combinations of exceptions, and wrotemacrosforall of these.
[17:56] <arigo> I believe it's overdone
[17:56] <stakkars> well, it was easy to write that way. I can just rename.
[17:57] <stakkars> The idea was that the C generator is abpletospit out exactly the checks that the code wanted it to do.
[17:57] <stakkars> able to spit
[17:57] <arigo> wait, first question
[17:57] <stakkars> so in other words, you don't wantall of these, just the checked versions, plus the ovf checked versions
[17:58] <arigo> there are several levels of optimizations, I think we've talking a bit past each other
[17:58] <arigo> did you implement this:
[17:58] <arigo> replacing from
[17:58] <arigo> add, ovfcheck ---- long handler -----> overflow handler
[17:58] <arigo> to
[17:59] <arigo> add_ovf ----> overflow handler
[17:59] <arigo> ?
[17:59] <stakkars> yes,yesterday it was this way.
[17:59] <arigo> ok
[17:59] <arigo> the "ovf" in add_ovf means specifically "check for int overflows too"
[17:59] <stakkars> Today I changed my mind and simplyremoved ovfcheck, becuase I thought I need todo more,anyway
[18:00] <arigo> no, there is confusion between OverflowError caught for float overflows, and the same OverflowError raised by ovfcheck() or add_ovf
[18:00] <arigo> if you wanted to be more precise, you could rewrite instead to:
[18:00] <arigo> add_ovf ----> intoverflow handler
[18:01] <arigo> with a subclass IntOverflowError of OverflowError
[18:01] <arigo> then it would make sense to rewrite instead as
[18:01] <arigo> add ----> intoverflow handler
[18:01] <arigo> the 'ovf' wouldn't be needed because we can deduce it from the intoverflow handler.
[18:02] <stakkars> ok, that is not much of a difference.
[18:02] <arigo> yes.
[18:02] <arigo> I don't propose to do that
[18:02] <stakkars> but I see I did **way** too much
[18:02] <arigo> yes
[18:03] <arigo> the code generators *need* to know about this add_ovf which does more than 'add'
[18:03] <stakkars> this is also,because I needed to rewrite the complicated handler for that function call.
[18:03] <arigo> but then they can look themselves to see what exceptions are caught after an 'add' or 'add_ovf'
[18:03] <stakkars> So I wrote something that goes for all the cases and produces nice exitcases.
[18:03] <arigo> I see
[18:04] <arigo> but in this case we are only interested in the OverflowError
[18:04] <arigo> any other exception should be forgotten
[18:04] <stakkars> wot? no
[18:04] <arigo> they would be an artefact of the ovfcheck() call appearing in the source code inside a handler
[18:04] <arigo> inside a try: except SomeOtherError:
[18:05] <stakkars> you have
[18:05] <stakkars> try: ovfcheck(a<<b)
[18:05] <stakkars> and then
[18:05] <stakkars> except OverflowError:
[18:05] <stakkars> except ValueError:
[18:05] <stakkars> etc.
[18:05] <stakkars> I wanted them both as exit cases.Works great.
[18:05] <arigo> but in this case ovfcheck() cannot really jump to the ValueError handler
[18:06] <stakkars> does not matter.
[18:06] <arigo> does :-)
[18:06] <stakkars> wait
[18:06] <stakkars> OverflowError is added to the can_raise things of the operator.
[18:06] <stakkars> By that, both the operator and ovfcheck get the needed handlers.
[18:07] <arigo> more precisely:
[18:07] <arigo> the << gets the OverflowError handler,
[18:07] <stakkars> I simply remove the ocfcheck call and remove the exits which are notNone, and we are ready.
[18:07] <arigo> while ovfcheck gets both handlers
[18:07] <arigo> stakkars: no
[18:07] <stakkars> not true, bot get them.
[18:07] <arigo> wait
[18:08] <arigo> it's again the confusion between FloatOVerflowError and IntOverflowError
[18:08] <arigo> consider <<
[18:08] <arigo> stakkars: uh?
[18:08] <arigo> if you take the flow graph of
[18:08] <arigo> try: ovfcheck(a<<b)
[18:08] <arigo> except OverflowError:
[18:08] <arigo> except ValueError:
[18:09] <arigo> then the << operator doesn't get the ValueError handler
[18:09] <stakkars> sure it does.
[18:09] <arigo> I must see it with my eyes.
[18:10] <stakkars> that's the trick, I can simplydrop the check stuff with handlers, and it's fine.
[18:10] <pedronis> ovfcheck(<<) should lshift_ovfcheck
[18:10] <arigo> no it doesn't.
[18:10] <arigo> wait
[18:10] <arigo> if you just take the flow graph of the above function,
[18:10] <arigo> and do no transformation,
[18:11] <arigo> then << doesn't have the ValueError handler.
[18:11] <pedronis> stakkars: is that from existing code?
[18:12] <arigo> stakkars: we are mixing current behavior, my expected behavior, and yours, which are 3 very different things :-)
[18:14] <stakkars> le tme check, it is hard because I have so many things open...
[18:18] <stakkars> the simplified, un-annotated graph starts with the operation
[18:18] <stakkars> in my case it is mod(x,y)
[18:19] <stakkars> this first block has three exits. One regular and twofor the ZeroDivision and Overflow
[18:19] <arigo> what is your exact source code?
[18:19] <stakkars> the None case continues to the ovfcheck function, which doess all its complication.
[18:20] <stakkars> just added to the snippetsbut not checked in:
[18:20] <stakkars> def mod_func(i=numtype):
[18:20] <stakkars> try:
[18:20] <stakkars> return ovfcheck((-maxint-1) % i)
[18:20] <stakkars> except OverflowError:
[18:20] <stakkars> raise
[18:20] <stakkars> except ZeroDivisionError:
[18:20] <stakkars> raise
[18:20] <arigo> ok, that's quite different from what you pretended above
[18:20] <arigo> ValueError is not an implicitly raised exception by <<
[18:20] <stakkars> not at all.
[18:21] <arigo> please read again what I said
[18:21] <arigo> with my small code example you don't get a ValueError handler for the first operation
[18:21] <stakkars> sure ValueError is implictly raised.
[18:21] <arigo> please try my small example
[18:21] <stakkars> we are wasting time.
[18:22] <stakkars> a << b raises ValueError for negative shifts.
[18:22] <arigo> <stakkars> you have
[18:22] <arigo> <stakkars> try: ovfcheck(a<<b)
[18:22] <arigo> <stakkars> and then
[18:22] <arigo> <stakkars> except OverflowError:
[18:22] <arigo> <stakkars> except ValueError:
[18:22] <arigo> <stakkars> etc.
[18:22] <arigo> <stakkars> I wanted them both as exit cases.Works great.
[18:23] <pedronis> but we cannot have << in a ovfcheck in our code
[18:23] <arigo> i'm just saying that the graph of this precise example doesn't have a handler for << ValueError on my machine.
[18:23] <stakkars> sorry, sorry, I took the wrong example because here we have the totally different case of ovfcheck_lshift.
[18:24] <stakkars> Or let's rewrite the logs to what we meant :-)
[18:24] <stakkars> ignore the shift problem at all, modulo is ok.
[18:24] <arigo> what I wanted to say at the beginning what this:
[18:24] <arigo> try:
[18:25] <arigo> ovfcheck(a%b)
[18:25] <arigo> except OverflowError: raise
[18:25] <arigo> except ValueError: raise
[18:25] <arigo> in this case, the ValueError case doesn't appear as a handler to the mod
[18:25] <arigo> it appears at the end of ovfcheck() but it's an artefact
[18:25] <arigo> it should *not* be added as a handler to the mod
[18:26] <stakkars> without further simplification I think ValueError would appear in both operations.
[18:26] <arigo> please try it.
[18:26] <stakkars> Afterwards, wego through thecan_onlystuff, and the thing gets dropped.
[18:27] <stakkars> ok, sure you win. It is not in flow/objspace, so it won't get generated.
[18:27] <arigo> yes
[18:27] <arigo> that was my point all along... sorry
[18:28] <stakkars> this word would have saved a lot,sorry too :-)
[18:28] <arigo> ValueError isn't in the flow objspace for << either, at the moment (which may be a bug but let's not discuss this now)
[18:28] <stakkars> ok, whether or not, dropping the ovfcheck thing is just fine
[18:29] <stakkars> sorry again, I added that.
[18:29] <stakkars> I shoudl have checked in, earlier, but things went so crazy, I couldn't stop.
[18:30] <stakkars> back to the problem:
[18:30] <arigo> so all I wanted to ask is if, because you're listing all exit cases of the ovfcheck, you're adding them all to the 'mod' operation?
[18:31] <stakkars> yesterday, I just added a single OverflowError exit case.
[18:31] <stakkars> today, I even dropped that, becuase the case growsfrom alone.
[18:31] <arigo> right
[18:32] <stakkars> what I wanted to say: The OverflowError is nothing special but for the fact that we have
[18:32] <stakkars> to handle it for CPython.
[18:32] <stakkars> For the generated code, ovfcheck can be just a no-op, becuase the relevantoperation
[18:32] <stakkars> already has the handler, if it needs it.
[18:33] <stakkars> (provided it is added to the correctlists, which was a while of debugging)
[18:34] <stakkars> so if you want me to rewrite things before annotation
[18:34] <arigo> (no, no longer)
[18:34] <stakkars> then I must assume that every operation has special cases handled,even if there is no handler?
[18:35] <arigo> I'm just confused now
[18:35] <arigo> of course the 'mod' operation always has the overflow handler already, because
[18:35] <arigo> we said that we must write ovfcheck(a%b)
[18:36] <arigo> so there is no way the 'mod' can be outside a handler while the ovfcheck is inside
[18:36] <stakkars> yes, that's the nice thing about it.
[18:36] <arigo> so I'm just confused about why we did all this at all...
[18:36] <stakkars> all what?
[18:37] <arigo> all, basically
[18:37] <arigo> we could just ignore ovfcheck() in the backends, end of the problem
[18:37] <stakkars> the ovfcheck?
[18:37] <arigo> all == all the ovfcheck-related changes to the translator and the annotator
[18:38] <stakkars> well, it's a bit more. for ovfcheck_lshift, I had to replace the operation instead
[18:38] <stakkars> and in this case, some cleanup of the exception handlerstill makessense.
[18:38] <arigo> I see
[18:39] <arigo> you still have to dig for the overflow case
[18:40] <stakkars> I can simply use the handler that is in place, but this isn't nice code.
[18:40] <arigo> yes
[18:40] <arigo> and the can_only_raise attributes have a similar optimization-only role, then
[18:40] <stakkars> in a sense, yes.
[18:41] <arigo> they are nice but they are not as essential as I misleadingly thought for this problem
[18:41] <stakkars> But how do I filter the exceptions in the case of lshift?
[18:41] <arigo> ah
[18:41] <stakkars> that was what made looking onto this attr so attractive.
[18:42] <arigo> how so?
[18:49] <stakkars> in this case, I don't have the native exit cases, because there is no operator.
[18:49] <stakkars> well, just taking and reshuffling them would do.
[18:50] <arigo> ah, you wanted to peek only in the lshift.can_only_raise?
[18:50] <stakkars> yes, that looked tempting.
[18:50] <arigo> you could use the list implicit_exceptions['lshift'] instead
[18:50] <arigo> from the flow objspace
[18:51] <stakkars> yes, good idea.
[18:52] <arigo> ok sorry I didn't follow you here previously.
[18:52] <stakkars> that means: the ovfcheck.xxx fiddling goes into simplify [ it's ok, I worked on this for days while you did urgent stuff ]
[18:52] <arigo> ok
[18:53] <stakkars> and besides of a bit reshuffling, there is nothing to do at all but drop the calls.
[18:53] <stakkars> later in the annotator, the filtering against can_only_throw is done,
[18:53] <arigo> it seems so
[18:54] <stakkars> and then the problem comes (finally what I wanted to know in the first place):
[18:54] <stakkars> where do I rewrite operations with exception cases, and which do I rewrite?
[18:55] <stakkars> you said,overflow only.
[18:55] <stakkars> that would mean that all default exception handling must be implcitlydone.
[18:56] <stakkars> ah, ok, if you stick with this, then I *must* rewrite the ovf thing during simplify.
[18:56] <arigo> no, I'm now thinking about not rewriting at all (before CTyper at least)
[18:57] <arigo> it seems that the graph you obtain by just dropping ovfcheck and fiddling with ovfcheck_lshift, is rather nice
[18:57] <stakkars> ok. Makes sense, too, if we say that the attached exitcases define the flavor of the operation.
[18:57] <arigo> yes
[18:57] <arigo> it's a bit of a coincidence that the attached overflowerror is the correct one, but it's fine
[18:58] <arigo> then it's a "internal" question for the CTyper if and how to rewrite the operations
[18:58] <stakkars> then, if ctyper does it alone, does it rewrite oper to int_oper_ovf_zer
[18:59] <arigo> it depends on what GenC supports,
[18:59] <arigo> but I guess it doesn't make sense to be too detailed
[18:59] <stakkars> it is just a few generated table entries, but well.
[19:00] <arigo> it's a matter of knowing what kind of C code we ultimately want
[19:00] idnar (mithrandi@idnar.user) joined #pypy.
[19:00] <stakkars> you mean it is ok to generate exceptions which are ignored, later?
[19:01] <arigo> I mean that exceptions are not really C
[19:01] <arigo> the CTyper should evolve in the direction explained on http://codespeak.net/pypy/index.cgi?doc/translation.html#c-level-operations
[19:02] <arigo> i.e. putting very C-ish operations into the graph
[19:02] <arigo> some much more concrete than 'int_div_ovf_zer'
[19:02] <arigo> something much more concrete than 'int_div_ovf_zer'
[19:03] <stakkars> sorry?
[19:03] <stakkars> first, the document does not at all talk about exception cases,
[19:03] <stakkars> and howmoreconcrete do you want to become than this?
[19:04] <stakkars> int_div does int devide with no checks,
[19:04] <arigo> indeed, I only wanted to point out how concrete the operations it talks about are
[19:04] <stakkars> int_div__zer does the same but checks for zero division, etc.
[19:05] <stakkars> currentlywe generate exceptions, but thisoculd easily be changed.
[19:05] <arigo> I wanted to say that if we stick to C-ish operations, we can do with several low-level operations instead
[19:05] <stakkars> going below the error checking, but expressing even that by operations?
[19:06] <arigo> yes
[19:06] <arigo> basically, I've already tried and trashed too many attempts to write a GenC supporting more than PyObjects
[19:06] <arigo> so I'd like to plan the next move a bit more
[19:07] <arigo> by writing all the operations we want in this document
[19:07] <arigo> and how they translate to C
[19:07] <arigo> until we have a pretty good idea about how we can generate them from the current graphs
[19:07] <stakkars> ok, but my simple operations are not that bad, and I'd like to finish *this* move.
[19:07] <arigo> sure
[19:07] <arigo> consider however,
[19:08] <arigo> that there is a fix needed in genc unrelated to the typer
[19:08] <arigo> because now, 'add' is not just PyNumber_Add for PyObjects if it's followed by an overflow handler
[19:08] <arigo> this is something that must be taken care of even if the CTyper isn't run at all.
[19:10] <stakkars> this doesn't fit my brain, now.
[19:10] <stakkars> I would like to check in my C header files and ask you to have a look,
[19:10] <stakkars> which operations should be used right now.
[19:11] <arigo> ok
[19:12] <arigo> I'm saying: "did you consider what occurs if ovfchecks are dropped, but the CTyper isn't run?"
[19:13] <arigo> in other words: does ovfcheck(a+b) produce correct code with no annotation, no CTyper, just the simplification?
[19:13] <stakkars> auuwa
[19:14] <stakkars> this would denaturate into a regular object call, and since we have no type info, it would behave wrong.
[19:14] <stakkars> checked in
[19:15] <arigo> yes. that's what is missing before "this" move can be considered done
[19:15] <arigo> adding CTyper support can be next move
[19:15] <stakkars> well, I never proposed this transformation before annotation, talking about "my"move:-)
[19:16] <arigo> right, ok :-)
[19:17] <stakkars> then we are back, there. The operations *need* to be rewritten, because integer objects don't
[19:17] <stakkars> overflow. But we don't know if we have integers at all.
[19:18] <arigo> there is a way without rewriting the operations themselves:
[19:19] <arigo> some more hacks in genc.py
[19:19] <arigo> along the lines of listing all the operations that have an overflow-checking version
[19:19] <arigo> in some dict
[19:19] <arigo> then if we see this operation
[19:20] <arigo> and if there is an ovf exit handler
[19:20] <arigo> then we write OP_XYZ_OVF() instead of OP_XYZ()
[19:20] <stakkars> that's doable, but still leaves the unannotated thing wrong.
[19:21] <arigo> not if we shift again what 'add' means
[19:21] <stakkars> you drive me crazy
[19:21] <arigo> and don't make overflows a special case
[19:21] <arigo> for 'lshift' for example, it's known to be allowed to raise ValueError
[19:22] <arigo> only if there is a handler
[19:22] <arigo> so it doesn't stretch the definition to say that 'lshift' or 'add' detect and raise overflows only if there is a handler
[19:22] <arigo> and treat overflows like the other exceptions again at this point.
[19:23] <stakkars> that turns the ovfcheck into syntactic sugar which makes tests run on CPython, nothing else.
[19:23] <arigo> yes,
[19:23] <arigo> that's quite in line with your remark that ovfcheck(a%b) always has the correct handlers on the % already.
[19:24] <stakkars> errhm no, it is not correct.
[19:24] <pedronis> but we may have code that does int arithmetic and float aritmethic inside a OverflowError handler
[19:25] <pedronis> and the int arithmetic would be wrap around under the current definitons if it's not ovfcheck-ed
[19:25] <arigo> pedronis: I don't really want to think about this case :-)
[19:25] <stakkars> my point is:
[19:27] <stakkars> whatever definition add has, without type info, how do we decide what an overflow is?
[19:27] <pedronis> arigo: but we have again the risk of different behavior before and after translation
[19:27] <arigo> pedronis: yes, but it's a quite small risk
[19:28] <arigo> stakkars: do you mean, int overflow versus float overflow?
[19:28] <arigo> pedronis: to avoid that we really need a new IntOverflowError, fish the overflow handler from the ovfcheck() call, and stick it as an IntOverflowError back to the operation
[19:29] <stakkars> if we have no annotator, then the generated code must decide at runtime,
[19:29] <arigo> pedronis: looks a bit unnecessary, I'd rather add a note in the definition of ovfcheck
[19:29] <stakkars> that if the thing happensto be an int, it must overflowor not,depending on the handler?
[19:30] <arigo> stakkars: yes, but you can also say it "if there is a handler, then at run-time we check if it's an int and if it's overflowed"
[19:30] <arigo> stakkars: which is ok
[19:31] <stakkars> ah. yes, makes sense.
[19:37] <stakkars> please decide, I will go for some food, back in 25 min
[19:39] <arigo> well, it makes sense yes.
----- silence for 30 minutes ----- [20:09] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) joined #pypy.
[20:15] Action: stakkars is back
[20:16] <stakkars> after some food and some re-thinking, I have the impression
[20:16] <stakkars> that the original idea wasn't that bad:
[20:16] <stakkars> ovfcheck is only there to declare that we want an overflowing integer,nothing else.
[20:17] <stakkars> so it is of courseok to rewrite exactly these operators to operator_ovf, in simplify
[20:17] <stakkars> and I do nothing else, there.
[20:18] <stakkars> and GenC only gets these flavors, all other checks are enabled by default.
[20:20] <stakkars> I even can undo my additions concerning can_only_raise, because we really get the extra operators,
[20:20] <stakkars> which then have the attribute.
[20:20] <stakkars> arigo: just trying a simple thing which gets us tosomething.What do you think?
[20:31] <arigo> yes, that makes sense
[20:32] <stakkars> you saw that I overdid alittle with the C macros, I willrename them and just select ovf/not ovf,ok?
[20:32] <stakkars> at a later step, things can be further broken down.
[20:33] <stakkars> (I think I did this to get rpystone as fast as possible :-) )
[20:33] <arigo> makes sense
[20:33] <arigo> I mean, this breaking down makes sense,
[20:34] <arigo> but it should probably be selected by GenC or the CTyper,
[20:34] <arigo> and not before.
[20:35] <stakkars> yes. I see that this is a syntactic thing what behavior of the ints we want.
[20:36] <arigo> maybe.
[20:36] <arigo> I'm still not convinced that the presence of the overflow handler isn't good enough.
[20:39] <stakkars> well, that depends.
[20:40] <stakkars> assume you did something wrong, and you overflowed into lo long already, before
[20:40] <stakkars> you enter the "protected" region of the ovfcheck handler?
[20:41] <stakkars> to get the same semantics, we would needto define that the operation must overflow,
[20:42] <stakkars> even if the arguments are longs, already.
[20:42] <stakkars> I guess that was crap.:-)
[20:44] <stakkars> forgot that we are in RPython...
[20:53] <stakkars> about good enough: in fact it is quite likely that you wantto check for other overflows, but most of the time
[20:53] <stakkars> not overflows of the integers, which are for instance used to build other objects or such.
[20:54] <stakkars> after all, the r_int was not *that* bad in this respect.
[20:59] <arigo> stakkars: yes but the fully annotated code knows if it's int or float
[21:00] <arigo> ...grumble.
[21:00] <arigo> the can_only_raise mecanism was useful, but now it would remove the interesting overflowerror
[21:01] <arigo> it's still very useful, it removes a lot of crap
[21:01] <arigo> so you're just right
[21:01] <arigo> we need a simplify before annotation which changes 'add' into 'add_ovf'
[21:02] <arigo> and 'add_ovf' has a different can_only_raise
[21:03] <stakkars> yes, this seems to be a slim solution.
[21:04] <stakkars> and I agree that it's doable without the extra opcodes, but you would have the side
[21:04] <stakkars> effect that array indexing with overflow handling would be messy.
[21:07] <arigo> ?
[21:10] <stakkars> I meant, if we didn't go for the extra _ovf opcodes, then the following code lines
[21:10] <stakkars> try:
[21:11] <stakkars> array1[i-1] += array2[i-1]
[21:11] <stakkars> except OverflowError:
[21:11] <stakkars> # tralala
[21:11] <stakkars> given that we have arrays of float which we want to check for overflow,
[21:12] <arigo> yes, that's perfectly right
[21:12] <stakkars> then we would get error handlers for the integer indexing as well.
[21:12] <arigo> so it makes sense to have the _ovf version
[21:13] <stakkars> howgh!
[21:13] <stakkars> that means I go back to the messy waynow and do this :-)
[21:14] <stakkars> yes! See here:
[21:14] <stakkars> ovfcheck(array1[i-1] + array2[i+1])
[21:15] <stakkars> would even work for integer array, without checking indices.
[21:16] <arigo> yes
[21:17] <stakkars> (stopping this soon)
[21:17] <arigo> indeed, the "messy waynow" is the safest thing
[21:17] <stakkars> messy in the sense that my code is almost 50 lines
[21:17] <stakkars> what I'm still wondering about:
[21:18] <stakkars> do we enforce that the ovfcheck is embraced by an exception handler?
[21:18] <stakkars> with all the fiddling,I could also grow the needed branches ad hoc [ duck ]
[21:20] <arigo> makes sense too
[21:21] <stakkars> it would raise what the operation raises. This is then almostlike a macro.
[21:21] <arigo> yes
[21:21] <arigo> actually:
[21:21] <arigo> maybe all the simplification isn't needed
[21:22] <arigo> because the annotator will then remove the stuff
[21:22] <arigo> with the can_only_raise attribute
[21:22] <arigo> maybe all you need to do is take the whole exception handler and stick it to the new _ovf operation
[21:22] <arigo> hum, it's messy if there are already handlers on the _ovf operation
[21:23] <stakkars> currently the annotator just checks the list of exit cases.
[21:23] <stakkars> and the handler for a function is the general mess.
[21:24] <arigo> ah, right, it probably wouldn't work out of the box
[21:24] <stakkars> I mean, at somepoint the simplification must be done. Maybe in fact that I need to do it in the annotator, with more effort than now.
[21:24] <arigo> possibly
[21:24] <stakkars> but that's aplan!
[21:24] <arigo> :-)
[21:24] <arigo> btw, unrelated:
[21:24] <arigo> we have a file in svn which has been checked in with Windows end-of-line
[21:24] <arigo> (not your fault)
[21:25] <arigo> could you run fixeol on it, to fix it?
[21:25] <arigo> if we do that from Unix machines we generate huge diffs
[21:25] <stakkars> nanu? I didn't add it. Who else uses winnows?
[21:25] <stakkars> which file?
[21:26] <arigo> lib-python/modified-2.3.4/test/test_codeccallbacks.py
[21:26] <arigo> it's from ale
[21:27] <stakkars> done
[21:29] <stakkars> collecting again, the semantics of can_only_raise is that only exactly these can be raised
[21:29] <stakkars> and I don't look into the classes, and I ignore general Exceptions like Exception if not mentioned?
[21:30] <stakkars> soI would just walk the messy exception chain, look for exact matches, and build my exit links from that?
[21:31] Action: hpk notices that fixeol seems to produce huge diffs on win32, nevertheless
[21:38] <arigo> stakkars: it's messy
[21:38] <arigo> if there is an Exception in the exit cases, it should always be followed
[21:40] <stakkars> yuck!
[21:40] <arigo> but there might be a way to tell the annotator that the exception value is then an instance of, say, the more precise ValueError
[21:40] <arigo> then it would flow this information forward, and the next "isinstance" checks would have a known result
[21:41] <stakkars> yes, that's what I'm doing already to find the OverflowError case in the ovfcheck handler.
[21:41] <stakkars> I meant, is it ok, to follow all these, using exact matches?
[21:42] <stakkars> ah,no, you say "flowing"... sory I was too quick
[21:42] <arigo> yes, I thought about normal flowing
[21:42] <arigo> you can't patch the flow graph at this time
[21:43] <stakkars> what if I say
[21:43] <stakkars> try:
[21:43] <stakkars> 1 / 0
[21:43] <stakkars> except: something
[21:43] <stakkars> should it catch ZeroDivision?
[21:44] <stakkars> I'm talking about the implicit exceptions,all the time, btw.
[21:45] <arigo> yes, 'div' should have ZeroDivisionError as an implicit exception (if that's your question)
[21:45] <stakkars> my question is if the above construct would catch it?
[21:45] <arigo> ah, sorry
[21:46] <arigo> yes
[21:46] <stakkars> I guess it would not, today.
[21:46] <arigo> no, it does
[21:46] <arigo> (just don't try with constant 1 / 0 because the flow space complains about that)
[21:46] <arigo> remember that implicit exceptions are thrown by the flow space, which lets the interpreter catch them normally.
[21:48] <stakkars> ah, ok. flow space triggers them and sees whether the interpreter produces different paths
[21:49] <arigo> yes.
[21:49] <stakkars> and how comes thatsimple instructions create the nice exitcases,
[21:49] <stakkars> while a function generates a long chain of things?
[21:50] <stakkars> because we have to check for subclassness?
[21:50] <arigo> because a call generates an 'Exception' (as an implicit exception)
[21:50] <arigo> yes, so the interpreter does all the subclassing checks, and they are recorded
[21:51] <arigo> but if we raise an implicit exception of type Constant(ValueError), then we get known exact matches
[21:51] <stakkars> is Exception special cased, or would that happen with sub-Exceptions, too, if they are not exact?
[21:51] <arigo> there is a bit of special-casing
[21:51] <arigo> basically assuming that there is not too much inheritance going on, apart from in the Exception case
[21:52] <stakkars> so I needtoleave the Exception branch in, whatever can_only has tosay.
[21:53] <arigo> I think so yes.
[21:54] <stakkars> then, when I adjust implicit exceptions or can_only_raise against the chain of
[21:55] <stakkars> comparison that the ovfcheck handler has generated,
[21:55] <stakkars> do I search for exact matches,or do I ask for subclassness (well,and then I need to have code, again)
[21:56] <stakkars> probably best to reallyleave the handler in place...
[21:56] <arigo> probably
[21:56] <stakkars> ha!
[21:56] <arigo> then the _ovf operation would be protected by a single "Exception" handler, usually
[21:57] <stakkars> exact matches go into the exitcases
[21:57] <stakkars> and I prune the according branch from the handler.
[21:57] <stakkars> at the end, it is either empty, or the rest is attached as the last case.
[21:58] <stakkars> pruning works great: remove operations and exit cases, the rest gets cleaned up by exiting code.
[21:58] <arigo> good!
[22:00] <stakkars> ok, going home for that.
[22:01] <stakkars> thanks for the lot of time and info!
[22:01] <arigo> :-)
[22:02] <stakkars> bye
[22:03] <arigo> good night
[22:18] aleale (~chatzilla@0x535ca5a5.hrnxx9.adsl-dhcp.tele.dk) left irc: "Chatzilla 0.9.68a [Firefox 1.0.3/20050414]"
[22:18] Action: hpk looks at fixing test_global.py (after posting to python-dev, ugh :-)
[22:29] <hpk> arigo, pedronis: does it make sense to try to forward all impl-level warnings to the warnings applevel module?
[22:30] <arigo> if you manage to reasonably easily, yes
[22:30] <hpk> i give it a try, not sure it can work (i haven't used the warnings module much yet)
[22:31] <arigo> well ideally you should only forward during the compile() call
[22:31] <arigo> maybe not all the time
[22:31] <hpk> makes sense, probably
[22:32] <hpk> running the core tests with -T 1500MP takes almost a day on codespeak (and i have it in a daily run script :-)
[22:32] <hpk> so PyPy is not allowed to get slower, obviously
[22:34] <hpk> arigo: you didn't actually fix py.py's sys.path issue, did you?
[22:34] <arigo> no
[22:35] <arigo> hpk: be careful, a bit slower and you get infinite loads :-)
[22:36] stakkars (~tismer@rosine101.inf.fu-berlin.de) left irc: Read error: 113 (No route to host)
[22:36] Action: hpk issues py.test -T10000MP test_exceptions.py
[22:36] <hpk> ten giga-pystones against the last timeout error, whoa
[22:37] <arigo> well, maybe a quick look in test_exceptions could help figure out a number that we can reduce?
[22:37] <hpk> nah
[22:38] <hpk> first i want to throw a processor against it
[22:39] <hpk> hum, it is counting to maxint
[22:39] <hpk> on a 64bit machine
[22:39] <hpk> no, that is not the problem
[22:39] <hpk> the problem is again the warnings module
[22:47] <hpk> arigo: i am not sure that extending fake.py further and further is a good idea ...
[22:47] <arigo> hpk: sure
[22:48] <arigo> we could have said that test_cfgparser.py is not a core-enough test
[22:48] <hpk> actually, now that we have saner int-handling we could try to re-instate Michael Chermside's _sre impl
[22:48] <arigo> yes, that makes more sense
[22:49] <hpk> i think have to monkey-patch the interp-level warnings module :-(
[22:50] <hpk> it'S just not flexible enough to allow me to specify a callback
[22:50] <arigo> bouh
[22:52] <hpk> hum, that wouldn't be too nicely translateable, would it?
[22:53] <hpk> i mean if i do it in try-finally
[22:53] <hpk> so i guess i have to do it globally
[22:55] <hpk> and then i don't have a space
[22:55] Action: hpk re-hums
[23:03] <arigo> anyway, that's a temporary hack
[23:03] <arigo> calls to the built-in compile() isn't translateable, essentially
[23:04] <arigo> 83.7% tests passing! great!
[23:07] <arigo> dinner time...
[23:07] arigo (~arigo@ratthing-b407.strakt.com) left irc: "bye"
[23:08] pedronis (pedronis@ratthing-b246.strakt.com) left irc: "Chatzilla 0.9.67 [Firefox 1.0.2/20050325]"
[00:00] --- Wed May 4 2005