[00:42] _hannes (sgylkd@i528C12F6.versanet.de) left irc: "utz utz utz"
[00:51] hpk_ (~holger_kr@mail.trillke.de) left irc: Read error: 148 (No route to host)
----- silence for 6 hr and 25 minutes ----- [07:16] idnar (mithrandi@idnar.user) left irc: Read error: 110 (Connection timed out)
----- silence for 1 hr and 6 minutes ----- [08:22] arre (ac@ratthing-b3fa.strakt.com) joined #pypy.
----- silence for 21 minutes ----- [08:43] idnar (mithy@idnar.user) joined #pypy.
----- silence for 3 hr and 9 minutes ----- [11:52] mwh2 (~user@modem-242.erendis.dialup.pol.co.uk) joined #pypy.
----- silence for 42 minutes ----- [12:34] adim (~adim@logilab.net2.nerim.net) joined #pypy.
[12:42] mwh2 (~user@modem-242.erendis.dialup.pol.co.uk) left irc: "bye"
----- silence for 28 minutes ----- [13:10] _hannes (qimzniss@i528C114B.versanet.de) joined #pypy.
----- silence for 20 minutes ----- [13:30] cfbolz (~bolz@zenon.physi.uni-heidelberg.de) joined #pypy.
[13:30] <cfbolz> hi!
[13:30] <cfbolz> hpk: you there?
[13:30] <hpk> yes
[13:31] <cfbolz> ok. How was the type of the tests in conftest determined?
[13:31] <hpk> heuristic and manually
[13:31] <hpk> why are you asking?
[13:31] <cfbolz> I found mistakes :-)
[13:32] <cfbolz> I'm correcting them.
[13:32] <hpk> sure, you might need an 'svn up', i have done stuff today
[13:32] <cfbolz> I've seen that.
[13:33] <cfbolz> I also wrote I file that contain the tests that fail for me, together with short comments.
[13:33] <cfbolz> Where do you think I should put it?
[13:33] <hpk> you could put that in the testmap as a '#' for example
[13:33] <cfbolz> Ok, but my comments are sometimes lengthy
[13:33] <hpk> inserting '#' lines
[13:34] <hpk> how lengthy?
[13:34] <hpk> can you restrict yourself to like 4 or 5 lines?
[13:34] <cfbolz> yes. would that be ok in the testmap?
[13:34] <hpk> if not then we might want to think about something else
[13:34] <hpk> i would say yes
[13:34] <hpk> hum
[13:34] <cfbolz> hum?
[13:35] <hpk> or: we could make a ReST file which contains the test types in some nicely parseable snytax
[13:35] <hpk> this way you can even make links and whatnot to describe problems
[13:35] <hpk> i think i like this idea
[13:35] <hpk> but i am in the middle of a different refactoring regarding conftest/test2
[13:35] <hpk> so for the moment just put them inline in the testmap
[13:36] <cfbolz> ok.
[13:36] <hpk> lib/test2 is just messy messy, btw
[13:37] <hpk> 60% of the modules in there are just plain broken (and apparently have been for a long time)
[13:37] <cfbolz> yes. But some of the tests there really do something sensible, e.g. test_itertools
[13:37] <hpk> i am going to move things into test2/attic and keep only what i think makes sense
[13:37] <hpk> then i mail pypy-dev and ask if anybody wants to save something from the attic, otherwise i kill it
[13:37] <hpk> cfbolz: i know
[13:38] <hpk> basically the current idea is:
[13:38] <cfbolz> well, I didn't. I figured it all out again and found the version in test2 afterwards :-)
[13:38] <hpk> yes, the idea is that you would run test_itertools.py still from lib-python2.3/...
[13:39] <hpk> but it would use a 'ModifiedUTTestModule' and actually take it from pypy/lib/test2
[13:39] <cfbolz> ok. makes a lot of sense.
[13:40] <hpk> the stuff in lib/test2 will by default only run against CPython
[13:40] <cfbolz> ?
[13:41] <hpk> this way we can make sure that no broken modules are in there
[13:42] <hpk> otherwise we could have modifications that don't even pass on CPython
[13:42] <hpk> i think modified regr-tests should basically continue to pass against CPython
[13:42] <hpk> clearer?
[13:43] <cfbolz> yes. got it now. I got confused because I somehow didn't notice the "by default" above.
[13:45] <cfbolz> There seam to be a lot of tests (at least 4 or 5) that fail because of one specific uncaught interp-level exception in objspace/std/fake.py
[13:45] <cfbolz> pypy.objspace.std.model.UnwrapError: calling <built-in function backslashreplace_errors>: cannot unwrap <UserW_ObjectObject() instance of <W_TypeObject(UnicodeError)>>
[13:45] <cfbolz> with different type objects
[13:54] <hpk> i see
[13:54] <hpk> however, i wouldn't neccessarily add to the fake mess
[13:55] <hpk> with the _sre_parse.py we went a different (currently switched off) route
[13:55] <hpk> it's in pypy/module and basically is a clean wrapper around the original cpython module
[13:56] <hpk> it's switched off because there were integer unwrapping problems but that was before the ovf/int refactoring
[13:56] <hpk> it is probably better to wrap modules with this style in order to completely get rid of fake.py
[13:56] <hpk> at least we discussed this on the Washington Pycon sprint, it wasn't a final discussion though
[13:57] <cfbolz> ok. I'll just add appropriate comments to conftest.
----- silence for 43 minutes ----- [14:40] <hpk> cfbolz: it would be good to add rev-numbers to the comments about the test, btw
[14:45] <cfbolz> Doing it. Is the rest ok?
[14:45] <hpk> sounds good
[14:45] <hpk> "good" :-)
[14:45] <cfbolz> :-)
[14:46] <cfbolz> I'm going to concentrate a bit on tests for the next time because this is easier to fit into the thin free time slots I have between the university stuff.
[14:47] <hpk> sure
[14:47] <hpk> makes sense, i noticed that amd64 gives different results, though
[14:47] <hpk> s/though/btw/
[14:48] <cfbolz> ouch! So I should add my arch too?
[14:49] <hpk> don't worry too much
[14:49] <hpk> by default, we can assume comments relate to x86/32 bit
[14:51] <cfbolz> ok
----- silence for 29 minutes ----- [15:20] stakkars (~tismer@rosine120.inf.fu-berlin.de) joined #pypy.
[15:23] <cfbolz> hi!
[15:23] <stakkars> hi!
[15:37] <hpk> hi
[15:48] <stakkars> hpk: Hi!
[15:49] <stakkars> I saw your checkins where you dropped app2interp_temp
[15:49] <stakkars> and replaced it by ApplevelClass
[15:49] <stakkars> are you sure this is what you want?
[15:50] <stakkars> the way I meant it was to use function applevel,
[15:50] <stakkars> and the correct class is figured out by the existence of a # NOT_RPYTHON comment.
[15:51] <hpk> i just wanted to be sure that for my test helper i don't get the interp-translation
[15:52] <hpk> but i see your point
[15:52] <stakkars> this was not clear to me, since you also added that comment. :-)
[15:55] arigo (~arigo@vicky.ecs.soton.ac.uk) joined #pypy.
[15:56] <cfbolz> hi
[15:56] <arigo> hi!
[15:56] <stakkars> hi
[15:58] pedronis (pedronis@ratthing-b246.strakt.com) joined #pypy.
[15:58] <arigo> hi!
[15:58] <hpk> hi armin
[15:59] <hpk> hi samuele, rehi everybody
[15:59] <pedronis> hi
[15:59] <arigo> crowded :-) hi all
[16:00] Action: hpk is increasingly unhappy with lib/test2
[16:08] <pedronis> arigo: I'm looked only at the code of canonlyrase testearday also because at some point codespeak svn was down, I'm running the tests now and see how to fix them
[16:08] <pedronis> s/I'm/I have
[16:11] arre (ac@ratthing-b3fa.strakt.com) left irc: Remote closed the connection
[16:11] <pedronis> arigo: mmh, the problem seem deep not shallow
[16:11] <pedronis> with the tests
[16:13] <arigo> I first don't understand why it seems that implicit exceptions don't disappear
[16:15] <arigo> ah, I see the deep problem in question
[16:15] <pedronis> well, we don't schedule the exception handling blocks
[16:15] <pedronis> it seem
[16:15] <arigo> yes
[16:15] <stakkars> please share your insight :)
[16:16] <pedronis> because it is done in flowin and we bail out
[16:16] <arigo> stakkars: we made some experiment yesterday, which now sits in a branch, and we're looking at the flow graphs
[16:16] <pedronis> stakkars: we have progressed a lot in annotating all of PyPy
[16:17] <pedronis> we are trying to get rid of the last 3 blocked blocks
[16:17] <stakkars> Great!!. Is it ok to check into the main stuff?
[16:17] <arigo> the branch is not where we progressed well :-)
[16:17] <arigo> it's where we tried something and it's not working so far
[16:18] <arigo> the trunk works quite well already.
[16:18] <stakkars> ok
[16:18] <pedronis> yes, the branch is about getting rid of the 3 blocks which are essentially spurious
[16:18] <pedronis> but removing them is a bit involved
[16:18] <arigo> pedronis: in theory we have the same problem with BlockedInference, no?
[16:18] <arigo> for example:
[16:18] <arigo> try:
[16:18] <arigo> x = lst[index]
[16:18] <arigo> except IndexError:
[16:18] <arigo> ...
[16:19] <stakkars> (I've worked quitea bit on interpleveling the sys fiel stuff, will tryto merge)
[16:19] <arigo> and we know that lst is empty
[16:19] <pedronis> yes
[16:19] <arigo> stakkars: nice!
[16:20] <arigo> pedronis: I'm trying to write a test for that
[16:21] <arigo> yes, we have a blocked block indeed
[16:21] <pedronis> it seems that we need to schedule the exception blocks no matter what
[16:22] <pedronis> althoug of course if they wrongly depend on some things that we may not have been computed it gets a bit messy
[16:23] <arigo> basically
[16:23] <arigo> if the inference is blocked on some operation "op"
[16:24] <arigo> if "op" is not meant to raise an exception, then we give up and it is an error (at least if it stays blocked)
[16:24] <arigo> if "op" can raise an exception, we should go on anyway in the exceptional path
[16:24] <pedronis> yes and probably use impossible values for the things that have no binding
[16:24] <arigo> this description includes the fact that simple_call() can always raise an exception.
[16:25] <arigo> ah I see, yes it's messy
[16:25] <pedronis> yup
[16:25] <arigo> grumble
[16:26] <pedronis> well, it is already messy in some sense as we did things
[16:26] <pedronis> just in a different way
[16:26] <pedronis> we propagate special impossible values etc
[16:26] <arigo> yes, what bites is the already-existing hack that an exception-catching branch has no value for the *last* result Variable
[16:27] <arigo> the quickest way around it is probably to say that
[16:28] <arigo> it is ok for switches on last_exception to have only exceptional branches out
[16:28] <pedronis> ? there is the None branch
[16:29] <arigo> well, it gets not annotated and removed in this example
[16:29] <pedronis> we don't schedule it
[16:29] <pedronis> yes
[16:29] <arigo> hum. BadlyDefinedSemanticsError
[16:31] <pedronis> well, we need some code in flowin distinguishing cases
[16:31] <arigo> I'm worrying about the semantics of the flow graph model
[16:34] <arigo> if we say that it's ok to not have the None branch any more,
[16:35] <arigo> then we have a problem with simple_call(), where we basically have neither the None nor any other exception branch
[16:36] <arigo> one way around is to always have the non-exceptional branch around, put pointing to a "raise SystemError" equivalent
[16:36] <pedronis> yes the question is how to patch the resulting graph
[16:36] <pedronis> but indeed things are messy
[16:37] <pedronis> the issue is that you cannot patch the block terminating on a exception exit itself
[16:37] <pedronis> but need to patch its None exit etc
[16:37] <arigo> yes
[16:38] <arigo> the "Haskell" way of having "impossible" code paths would be as an infinite empty loop :-)
[16:38] <arigo> but a "raise SystemError" is probably much more helpful in case of problem
[16:39] <pedronis> yes
[16:39] <pedronis> but indeed we have cases were the raise can be added to the block
[16:39] <pedronis> and when it needs to be added as the non exceptional exit
[16:40] <arigo> well, that's not really a problem actually
[16:40] <arigo> transform_dead_code() in the branch
[16:40] <arigo> is already doing that
[16:40] <pedronis> I saw that
[16:40] <arigo> although there is an XXX
[16:40] <arigo> two XXXs even.
[16:41] <pedronis> yes
[16:41] <pedronis> and it needs to be careful about exception exits
[16:41] <arigo> yes, it's not complete right now, as it will happily kill the None exit
[16:42] <pedronis> as, I don't see problems just that we will need some amount of case handling both in flowin and in transform
[16:42] <arigo> ok
[16:47] <arigo> lemme try...
----- silence for 18 minutes ----- [17:05] <arigo> yes, seems to work
[17:06] <arigo> I also added a test file that does the same as test_annrpython but simplifies the graph afterwards, to check transform.py and to check that the graph are valid
[17:10] <arigo> it makes genllvm unhappy, though
[17:11] <cfbolz> what's happening?
[17:12] <arigo> test_pyrextrans isn't happy too
[17:12] <arigo> let me see
[17:14] <arigo> IntRepr.get() asks for get.annotator.binding(Constant(last_exc_value))
[17:14] <arigo> hum.
[17:14] <arigo> my bet is that the graphs are really incorrect
[17:15] <arigo> checkgraph() doesn't look for Constant(last_exc_value) that would be at incorrect places
[17:15] <cfbolz> well, could be a problem of the get, too. Some get methods are a bit messy
[17:16] <arigo> given that pyrextrans has troubles too, I guess it's rather the graph that are broken or at least unexpected
[17:17] <cfbolz> ok
[17:25] <arigo> ah oh
[17:26] <arigo> I get confused: when are implicit exceptions supposed to be removed?
[17:26] <pedronis> arigo: did you check in a fix?
[17:27] <arigo> yes, everything's checked in the branch
[17:27] <arigo> uh
[17:27] <arigo> t.simplify() must be called *twice* in the branch to remove the implicit exceptions
[17:28] <arigo> so the genllvm problem is just that it sees unexpected extra exceptions all around
[17:28] <cfbolz> time for a simplification pass manager :-)
[17:29] <arigo> uh, same problem in the trunk
[17:29] Action: arigo is confused
[17:29] <arigo> does llvm work right now in the trunk?
[17:30] <pedronis> I don't know
[17:30] <arigo> let me check
[17:30] <arigo> no, same problem
[17:30] <cfbolz> I can't check it out from here, I'll look after it tonight. What's failing?
[17:31] <arigo> well, lots of tests, the first one at least with the error I described above
[17:31] <cfbolz> thanks.
[17:31] <pedronis> what change can have broken it?
[17:32] <arigo> I'm doing a binary search to know that :-)
[17:33] <arigo> it's revision 10842
[17:34] <pedronis> ?
[17:34] <arigo> ? indeed
[17:34] <arigo> yes, it really is this revision
[17:35] <cfbolz> huh?
[17:35] <pedronis> but it changes exception tracing in the pypy interpreter (and fixes it)?
[17:36] <arigo> well, it doesn't make sense to me so far, but at least there is the word "exception" in the check-in message
[17:37] <arigo> aArgh I know.
[17:37] <arigo> you added a space.newtuple()
[17:37] <arigo> so the implicit exception branch isn't empty any more
[17:38] <pedronis> I see
[17:38] <pedronis> that step need to be lazy
[17:39] <arigo> yes
[17:39] <pedronis> I can fix that
[17:40] <pedronis> we need to merge
[17:40] <arigo> ok
[17:40] <arigo> I can merge the branch back now, if it's ok for you
[17:40] <arigo> :-)
[17:40] <pedronis> yes
[17:40] <pedronis> of course there have been checkins on trunk
[17:40] <pedronis> on the trunk
[17:41] <arigo> yes, but I'll just merge "normally" with svn merge
[17:41] <pedronis> makes sense
[17:41] <arigo> there is no interesting check-in message in the branch, better see a single interesting check-in on the trunk with a good message
[17:42] <pedronis> btw, there's still the change about isinstance(.,long) on the branch
[17:42] <arigo> it should go away if it's been fixed in the trunk after the fork
[17:43] <arigo> the fork is from rev 10864
[17:43] <pedronis> no, but we discussed that what was going on is correct
[17:43] <pedronis> and can rechange that and add a comment about that
[17:44] <arigo> ah yes I remember
[17:45] <pedronis> the type(s_obj) < SomeObject -> False
[17:46] <pedronis> type(s_obj) == SomeObject -> SomeBool
[17:46] <arigo> basically I need to revert annotation/builtin.py
[17:46] <pedronis> yes
[17:46] <pedronis> and then I will add a comment on the trunk
[17:47] <pedronis> because one is needed
[17:47] <pedronis> and fix the laziness issue
[17:48] <arigo> a comment about annotation/builtin.py ?
[17:48] <pedronis> about why doind r.const = False doesn't violate the invariant
[17:52] <arigo> ok.
[17:52] <arigo> ok merged
[17:55] <stakkars> I just noticed that test_all.py modulename doesn't work any longer.
[17:56] <stakkars> what is the new way to run one specific test?
[17:56] <arigo> test_all.py <filename> should work
[17:57] <stakkars> without giving the path, maybe?
[17:57] <arigo> no, <filename> as a path, relative or absolute, or a directory name for all files within
[17:57] <stakkars> ahh, I see. The -S option is gone.
[17:57] <arigo> it's called -s now
[17:57] <pedronis> it's -s
[17:57] <stakkars> thanks!
[17:57] <arigo> all "built-in" test options have been lower-cased.
[17:57] <stakkars> ok.
[17:58] <stakkars> One thing where I stepped upon:
[17:58] <stakkars> when running test_objspace.py in flow space, I got into trouble
[17:58] <stakkars> because flow objspace does not have space.newint and friends.
[17:58] <stakkars> I changed geninterp to avoid newint and newfloat, also because they are not listed
[17:59] <stakkars> in the interface string in defaultobjspace. Is that ok? (I mean, is that interface normative for all?)
[18:09] <pedronis> you were using newint(.) instead of wrap(.) ?
[18:09] <stakkars> yes, for a very long time I did :-)
[18:10] <stakkars> and this crashed, after a cache-compiled init function needed to initialize with slow space :-)
[18:10] <stakkars> s/slow/flow
[18:10] <arigo> "slow space" is the stdobjspace, not to be confused :-)
[18:11] <stakkars> yeah, thank you :-D
[18:11] <stakkars> let's go and rename it.
[18:11] <arigo> :-)
[18:12] <stakkars> I see that my cache file has grown substantially. I believe some more modules
[18:12] <stakkars> got into translation during my absence? (yes, I think I read all check-ins, but missed that)
[18:12] <stakkars> or I might have a bug, there...
[18:13] <pedronis> it could be my change
[18:14] <pedronis> wich I'm fixing
[18:14] <pedronis> they may grown a lot of tuple constructions :)
[18:14] <arigo> right
[18:14] <stakkars> I got something like 35 % cache growth.
[18:18] <stakkars> well, there are also quite some new tests, which may have triggered more code generation.
[18:18] adim (adim@logilab.net2.nerim.net) left #pypy.
[18:30] <pedronis> done
[18:31] <stakkars> but I also feel that eceptions aremore lengthy, in a way.
[18:34] <stakkars> we have a failing test in test_itertools.py
[18:34] <stakkars> same at your side?
[18:34] <arigo> ah, didn't try the new lib/test2 tests so far
[18:35] <arigo> cfbolz: genllvm works nicely again
[18:35] <arigo> thanks Samuele :-)
[18:36] <cfbolz> arigo: good
[18:36] <pedronis> OTOH targetpypy dies very soon now
[18:36] <arigo> stakkars: no, it works for me.
[18:36] <arigo> maybe you need to update in the parent directory to update lib-python-2.3.4 too?
[18:37] <stakkars> gargl
[18:37] <cfbolz> works for me too
[18:38] <stakkars> hmm, still not.
[18:38] <hpk> stakkars: which invocation exactly fails?
[18:39] <pedronis> arigo: targetpypy finishes very soon with 3 blocked blocks
[18:39] <stakkars> test_sf_950057(self):
[18:39] <arigo> hpk: for what it's worth: "py.test test_itertools.py" in lib-python-2.3.4/test is really slow and gives several failures
[18:40] <stakkars> maybe it breaks for you if you svn up, and I broke something? (but I don't think so, disabled geninterp)
[18:40] <arigo> stakkars: how do you run the test?
[18:41] <stakkars> D:\pypy\dist\pypy>test_all.py lib\test2\test_itertools.py
[18:41] <arigo> ah? yes I see the same failure when I run it like this
[18:41] <stakkars> looks like some path problem
[18:42] <arigo> yes I guess that if you run ..\..\test_all.py test_itertools.py it works?
[18:42] <stakkars> no, fails too
[18:43] <arigo> hum for me too
[18:43] <stakkars> funny
[18:43] <arigo> but not if I run py.test test_itertools.py
[18:43] <arigo> conftest.py fun
[18:44] <arigo> argh. side note: why is itertools.py implemented with classes instead of just generators?
[18:45] <hpk> ASFAIK (i am not the implentor), the implementation was taken from something that Raymond wrote
[18:45] <pedronis> wasn't it that way. maybe some tests break otherwise
[18:46] <arigo> yes, I start to see
[18:46] <stakkars> no idea.perhaüs somebody wanted to make it translatable?
[18:46] <arigo> e.g. you want islice(lst, step=0) to raise ValueError as soon as it's called
[18:46] <hpk> arigo, stakkars: i don'T get any failure with test_all.py and test_itertools.py no matter where i run it
[18:46] <arigo> you can't do that with generators.
[18:47] <stakkars> hpk: your svn is updated? your changes all checked in?
[18:48] <pedronis> arigo: the targetpypy problem is very strange
[18:48] <pedronis> the graph of match_signature where we block
[18:48] <pedronis> has a lot of constants {} dicts instead of variables?
[18:49] <arigo> uh
[18:49] <hpk> stakkars: yes
[18:49] <cfbolz> stakkars: test_all.py and test_itertools.py works for me, too. And I did a recent svn up
[18:50] <arigo> rm *.pyc
[18:50] <arigo> doesn't help me, but maybe helps hpk and cfbolz to crash :-)
[18:51] <hpk> nope
[18:51] <arigo> ouack ouack ouack.
[18:52] <hpk> which python version, btw?
[18:52] <cfbolz> nope, too :-)
[18:52] <arigo> I get a different crash with 2.3 and 2.4.
[18:52] <arigo> but one crash exactly, in each case.
[18:52] <stakkars> Python 2.3.3 Stackless 3.1b3 050119
[18:52] <cfbolz> Python 2.3.4 for me.
[18:53] <arigo> 2.4.1 or 2.3.3
[18:53] <hpk> arigo: do you get "did not raise" in the 2.4.1 case?
[18:53] <arigo> hpk: yes
[18:53] <hpk> ok, i get it with 2.4 as well, but
[18:53] <hpk> it doesn#t make much sense to run a 2.3.4 test against 2.4 IMO
[18:54] <arigo> it shouldn't matter, should it?
[18:54] <hpk> no clue
[18:54] <hpk> maybe i should go ahead and disable running of tests against cpython anyway
[18:54] <stakkars> there *might* be some interaction between different python versions insalled in parallel
[18:55] <hpk> it's too flaky (especially if you use non 2.3.4 python versions)
[18:55] <stakkars> if something goes wrong with sys.path?
[18:55] <arigo> stakkars: I doubt it
[18:55] <arigo> hpk: shame
[18:55] <arigo> Ah no, I know
[18:56] <arigo> I think that our itertools.py is broken in a subtle way
[18:56] <arigo> at least it doesn't explicitely contain the test that this DID NOT RAISE checks for
[18:57] <hpk> no, i know :-)
[18:57] <arigo> ah
[18:57] <hpk> the thing is that test_itertools.py runs against our itertools.py implementation to begin with
[18:57] <arigo> I know
[18:58] <arigo> the bug is in our itertools.py, and it's fine to run it on various Python versions because it discovers such bugs, no?
[18:58] <stakkars> what's the bug?
[18:58] <arigo> well, islice(seq, 'a') doesn't raise the expected ValueError
[18:59] <arigo> (why it isn't a TypeError is beyond me)
[18:59] <stakkars> funny that this shows up now, not earlier.
[19:00] <hpk> i enabled the testing of test_itertools.py just today
[19:00] <arigo> apparently, it depends on a hard-to-reproduce test
[19:00] <arigo> it passes or fails depending on 'a' < 0
[19:01] <stakkars> eieiei
[19:01] idnar (mithy@idnar.user) left irc: Nick collision from services.
[19:01] idnar_ (mithrandi@idnar.user) joined #pypy.
[19:01] idnar (mithy@idnar.user) joined #pypy.
[19:03] idnar (mithy@idnar.user) left irc: Nick collision from services.
[19:03] <stakkars> more fun alert:
[19:03] <arigo> well I guess "someone" should review the whole of itertools for such problems
[19:04] <stakkars> test_rypstone behaves differently, depending on the absense of the -s flag???
[19:04] <arigo> pedronis: oups
[19:04] <arigo> pedronis: the graph we see for targetpypy has been rewritten by the C typer already
[19:04] <pedronis> yes
[19:05] <arigo> so Variables annotated as constants are replaced by real Constants.
[19:05] <pedronis> we should disable that maybe
[19:05] <pedronis> but it's something very strange going on
[19:05] <arigo> for this specific problem, I think it's just because it crashed early that the annotations are left incomplete.
[19:06] <pedronis> is it crashing?
[19:06] <stakkars> haaa, I'm obviously hitting an pdb.set_trace call which I use for some special cases,
[19:06] <stakkars> but this special case exactly doesn't show up when I use -s for testing :-( :-( :-(
[19:08] <pedronis> arigo: we are blocked in issubtype
[19:11] <arigo> pedronis: because we haven't seen any construction of FailedToImplement instances so far, I guess
[19:11] <pedronis> well, but then is match_signature
[19:12] <pedronis> or we are rescheduling something properly
[19:12] <pedronis> are not
[19:17] <arigo> get_and_call_function__4 is strange
[19:20] <arigo> I just added -no-s (don't call a.simplify()) and -no-t (don't call a.specialize()) to translate_pypy... should make more sense to debug a crashing annotation phase
[19:21] <pedronis> maybe we simply never reach code that creates Arguments with keywords
[19:22] <arigo> yes, I think so
[19:22] <arigo> with the -no-s -no-t options, it becomes easy to follow functions that don't return anything so far
[19:22] <arigo> Space.call_args is an interesting candidate
[19:25] <arigo> class BuiltinCode is missing the create_frame() method!
[19:25] <stakkars> help
[19:25] <arigo> there is just the abstract, "raise TypeError" base create_frame() method
[19:26] <arigo> which blocks everything.
[19:26] <pedronis> eh?
[19:26] <pedronis> BuiltinCode.create_frame is defined
[19:27] <arigo> uh, not on my side
[19:27] <arigo> ah oups yes
[19:27] <pedronis> in the graph or in teh code
[19:27] <arigo> ah, interesting
[19:28] <arigo> in the flow graph, after clicking on class BuiltinCode
[19:28] <arigo> I forgot that create_frame would be present in Code only
[19:28] <arigo> so indeed I see that Code.create_frame is a PBC with three choices
[19:28] <arigo> !
[19:28] <arigo> I know what's wrong.
[19:29] <arigo> method call --> three choice, one of which always raises an exception
[19:29] <arigo> recursivecall() throws a CanOnlyRaise
[19:29] <arigo> and the other 2 choices don't have a chance to provide an answer
[19:29] <arigo> bad bad bad idea, this CanOnlyRaise exception.
[19:31] <pedronis> ah argh
[19:31] <pedronis> I see
[19:32] <pedronis> but we would have the same problem with BlockInference
[19:33] <pedronis> but that never happened
[19:33] <pedronis> so we were just lucky
[19:33] <arigo> no,
[19:33] <arigo> because we didn't raise BlockedInference here,
[19:33] <arigo> we just returned a benign SomeImpossibleValue
[19:34] <pedronis> yes, but other situation could have thrown a BlockedInference
[19:34] <pedronis> and so we would have also skipped cases
[19:34] <arigo> ah yes, the hasonlyexcreturn() case
[19:35] <arigo> I think we need to return SomeImpossibleValue() cleanly
[19:35] <pedronis> and convert it
[19:35] <pedronis> to an exception
[19:35] <pedronis> some levels up
[19:35] <arigo> yes
[19:35] <arigo> actually
[19:36] <arigo> right now it will be turned into a BlockedInference
[19:36] <arigo> then the BlockedInference forces a check for "are we on an exception handler"
[19:36] <stakkars> found my problem:
[19:37] <stakkars> printing in test context writes to a sys.stdout that is a cStringIO object
[19:37] <stakkars> and that can't be handled, yet
[19:38] <arigo> pedronis: just checked in a test for the problem
[19:42] <cfbolz> I'm going home. See you.
[19:43] <arigo> see you!
[19:43] <pedronis> see you
[19:43] <stakkars> cu
[19:43] <arigo> pedronis: do you see a solution apart from reintroducing "benign" ?
[19:43] cfbolz (~bolz@zenon.physi.uni-heidelberg.de) left irc: "Leaving"
[19:43] <pedronis> well special casing call operations
[19:44] <arigo> simple_call and call_args
[19:44] <pedronis> but I'm looking at something else
[19:44] <pedronis> yes
[19:44] <arigo> ok
[19:44] <pedronis> and have them convert the impossible value to the exception
[19:45] <pedronis> is it always correct not to rest annotated[] on a BlockedInference
[19:45] <pedronis> if there are exception handlers
[19:45] <arigo> well, there might be no exception handler, but it's still correct
[19:46] <pedronis> what I'm saying is that the new code from the branch
[19:46] <pedronis> does not reset annotated if there are exception handlers
[19:46] <pedronis> because it doesn't rethrow the BlockedInference exception
[19:47] <arigo> ah, yes sorry
[19:48] <pedronis> that means the handler in processblock is not triggered anymore
[19:48] <pedronis> setting debug info etc
[19:48] <pedronis> is that correct?
[19:49] <arigo> maybe
[19:50] <pedronis> that's the question
[19:50] <arigo> we can set why_not_annotated in flowin(), to be on the safe side
[19:50] <pedronis> yes
[19:52] <pedronis> are you doing the changes or should I?
[19:52] <stakkars> got multiple cache entries because things are spawning subprocesses.
[19:53] <stakkars> will try to solve that...
[19:53] <pedronis> yes test_py.py does that
[19:53] <arigo> pedronis: working on it
[19:53] <stakkars> :-( will check the file size
[19:57] <arigo> checked in
[19:58] <arigo> tests pass (including the new test showing the problem) and targetpypy goes much farther, so it looks good.
[19:59] <arigo> pedronis: also note that why_not_annotated is pretty much useless now
[20:00] <arigo> because BlockedInference has become a local exception that can only be raised from consider_op()
[20:00] <arigo> so sys.exc_info() isn't interesting
[20:01] <pedronis> I see
[20:01] <pedronis> at some point we had stuff in annotation/ that did raise it
[20:01] <arigo> yes, the factory used it heavily
[20:01] <arigo> the next problem of targetpypy is (I think) about "def hole" returning None, but then None being unioned with something else
[20:02] <arigo> ah no, worse: even just unionof(None) crashes
[20:03] <pedronis> ah
[20:03] <pedronis> hole is really used for stuff
[20:03] <pedronis> that is just called without using the result
[20:04] <arigo> let's fix unionof() to ignore None input arguments, maybe
[20:04] <arigo> ah no
[20:04] <arigo> beuh
[20:04] <arigo> unionof(None) should return None
[20:05] <pedronis> yes
[20:05] <arigo> doesn't make much sense
[20:05] <arigo> unless we fix a lot of places to allow None through
[20:06] <pedronis> I think hole can return SomeImpossible value now
[20:07] <arigo> no, that would mean that fake_exception always raises an exception
[20:07] <arigo> if anything, it should return SomeNone
[20:07] <pedronis> SomePBC(None)
[20:07] <arigo> yes
[20:08] <pedronis> wich is what ignore does
[20:08] <arigo> SomePBC({None: True}) :-)
[20:08] <arigo> I see
[20:08] <pedronis> we can remove hole and just use ignore
[20:09] <arigo> if it's just for wrap_exception, it's kind of obvious that the result is not used
[20:09] <pedronis> yes
[20:09] <pedronis> anyway is not like we are left with a lot of stuff
[20:10] <pedronis> if targetpypy now works again
[20:10] <arigo> checking in...
[20:10] <arigo> ok
[20:10] <stakkars> OT:
[20:10] <stakkars> I have the problem of multiple cache entries due to recursive testing.
[20:11] <stakkars> what should I do: Ignore it, or try to reload the cache file if I detect this?
[20:11] <arigo> pedronis: btw, you know that you can run translate_pypy.py on a fast remote machine with a port number as argument
[20:11] <arigo> pedronis: and view the flow graph with a local viewer (yes I'm sure you do, actually)
[20:12] <pedronis> arigo: yes, but now my local machine is quite fast
[20:12] <arigo> great :-)
[20:13] <pedronis> it's 2.4Gh amd64
[20:13] <stakkars> meep
[20:13] <arigo> stakkars: yes yes :-)
[20:13] <arigo> stakkars: sorry, I don't know really
[20:13] <stakkars> you say it's up to what I like the best/easiest. thx
[20:14] <arigo> yes
[20:14] <arigo> though reloading is always dangerous, if I may venture an opinion :-)
[20:14] <stakkars> I thought to save the file size and seek before writing.
[20:15] <arigo> and truncate after writing?
[20:15] <stakkars> yes. I think this should serialize things.
[20:15] <arigo> in theory it's not safe
[20:15] <arigo> you could mess up the data if two processes write at the same time
[20:16] <stakkars> I don't handle parallelism at all. It is just about recursion
[20:16] <arigo> ah
[20:16] <arigo> but what about people that run two py.py in parallel?
[20:17] <arigo> I'm not sure they really deserve being shot at :-)
[20:17] <stakkars> some tests call pypy recursively, which then try to build stuff.
[20:17] <stakkars> hmm.
[20:18] <stakkars> move the cache into an usession, and trying to reuse it?
[20:18] <stakkars> what I'mdoing now is to generate source and to write out one big blurb.
[20:18] <arigo> At this point, running py.py doesn't create a usession dir...
[20:19] <arigo> locking ahead...
[20:19] <pedronis> I'm sure that it has happend to run more that one process with objspaces in parallel
[20:19] <stakkars> sure, it could. But I don't know if we want this
[20:19] <pedronis> we have just been lucky that the cache was not being changed likely
[20:20] <pedronis> but just used
[20:20] <stakkars> alternatively I couldgo the harder way and use a directory with hash-named entries. baah
[20:21] <arigo> that makes some sense, actually
[20:21] <arigo> because then you wouldn't have to completely load the big cache at start-up
[20:22] <stakkars> well, yes, but you'd use much more file actions and computation
[20:22] <stakkars> and you get non-obviously named files.
[20:22] <arigo> right, but not that much more
[20:22] <arigo> I wouldn't mind a directory 'cache' containing only files with 32-letter hex names :-)
[20:23] <pedronis> you can put a line in them that can be easely grepped
[20:23] <pedronis> if you need to know what they correspond to
[20:23] <stakkars> I also could prefix them with the module name where the source appeared in
[20:23] <arigo> makes sense too
[20:24] <arigo> then we can have an idea about what kind of code is in the cache by looking at the directory
[20:24] <stakkars> yes
[20:24] <stakkars> and the locking problem might go away,
[20:24] <arigo> yes.
[20:24] <stakkars> first because I can easily re-use files created by sub-processes,
[20:25] <stakkars> second because the files have exactly the same content, if they get written twice.
[20:25] <stakkars> maybe I should lock them, anyway.
[20:25] <arigo> I can't think of a case where you need to lock them when writing the same content
[20:26] <stakkars> right.worst caseis that I write it too often.
[20:26] <arigo> actually, you are only ever *creating* them, so you can write a temp file, then rename it
[20:26] <arigo> the rename is atomic
[20:26] <arigo> that's a kind of locking.
[20:26] <stakkars> sure
[20:27] <stakkars> but we stillwant the files to contain source code,not code objects, right?
[20:27] <arigo> for inspection I guess so
[20:27] <arigo> the current trick of writing a .py file and importing it to reuse the compiled .pyc file is nice :-)
[20:28] <stakkars> yes, I'd like to keep it. fine for debugging
[20:28] <arigo> (hum, I don't want to know if CPython is safe against two processes importing the same module and writing out the .pyc file in parallel)
[20:29] <arigo> I believe so, actually
[20:29] <stakkars> is it unsafe at all if they both write the same content?
[20:30] <arigo> don't think so.
[20:30] <pedronis> arigo: it's unsafe
[20:30] <arigo> pedronis: oh well.
[20:31] <pedronis> that's also one of the reason for the proposal of having a env var
[20:31] <pedronis> to specify a cache for pyc
[20:31] <stakkars> ???
[20:31] <pedronis> there's a PEP for that stakkars
[20:31] <arigo> an envvar in CPython you mean
[20:31] <pedronis> yes
[20:31] <pedronis> so that if users share a same set of py files
[20:32] <stakkars> always the same story. I want tosolve asmall problem andmove on, and what happend..... :-)
[20:32] <pedronis> each can have her cache
[20:32] <pedronis> that's why is always a good idea for python software to generare the pycs on installation
[20:34] <pedronis> targetpypy is working again up to annotation
[20:35] <stakkars> pedronis: great
[20:35] <stakkars> I see no hint in PEP 304 why the parallel write is unsafe
[20:37] <arigo> stakkars: it hints that reference [4] is about this subject
[20:37] <stakkars> yes,it talks about file corruption, but with no facts added
[20:39] <arigo> stakkars: the thread in question contains what might have been the first mail about PyPy :-)
[20:39] <arigo> http://mail.python.org/pipermail/python-dev/2003-January/032071.html
[20:39] jacob22 (~jacob@c-51c6e055.1321-1-64736c11.cust.bredbandsbolaget.se) joined #pypy.
[20:40] <arigo> but indeed, it doesn't seem to talk about the issue
[20:40] <arigo> hi jacob
[20:40] Nick change: jacob22 -> lac
[20:40] <arigo> hi lac :-)
[20:40] <lac> hi hi.
[20:40] <lac> is samuele around?
[20:40] <arigo> yes
[20:40] <pedronis> lac: hi, yes
[20:40] <lac> all the phones to strakt aren't working.
[20:41] <lac> is jacob there?
[20:41] <pedronis> yes
[20:41] <lac> can you have him phone me,or login here orsoemtnhing?
[20:42] <stakkars> arigo: sure I know that
[20:42] <arigo> stakkars: sorry, slight mistake
[20:42] <arigo> it's not actually in the very same thread, it's the next one.
[20:42] <stakkars> so itis
[20:44] <stakkars> read it all - no hint *why* there is clutter. humm
[20:45] <stakkars> I will go for a directory with files as we said.
[20:45] <arigo> yes, looks good.
[20:45] <stakkars> double-clicking one kills it, importing is fine.
[20:45] <stakkars> double-clicking __init__ kills 'em all.
[20:45] <arigo> :-)
[20:46] <stakkars> will be nice, since you can see how things are done in order.
[20:46] <stakkars> while I'm at it - are there preferences where to put it and under what name?
[20:47] <lac> thanks samuele.
[20:47] <arigo> in /pypy/_cache maybe
[20:47] <stakkars> ok, that's fine
[20:48] <stakkars> this directory will get checked in with an __init__.py and take care ofitself.
[20:48] <stakkars> well,is there a way to svn-ignore this whole folder, just not the __init__ ?
[20:48] <pedronis> lac: no problem
[20:49] <arigo> you can svn:ignore the directory '_cache' from the directory 'pypy', sure
[20:49] <arigo> or you can svn:ignore '*.py' in _cache if you prefer to check in _cache and __init__.py
[20:49] <stakkars> the svn ignore would also ignore changes to __init__ if it was checked in previously?
[20:50] <arigo> no
[20:50] <arigo> it only prevents the '?' from showing up, not modifications to checked-in files.
[20:50] <stakkars> so I can ignore the whole directory, if this is about additions, only.
[20:50] <arigo> you can ignore *.py, yes
[20:50] <stakkars> ok
[20:50] <arigo> if you ignore _cache from the parent but still check it in, I guess the ignore has no effect
[20:51] <stakkars> then we add an executable clcache.py in the folder we aremost of the time, too (dist/pypy for my case),saves you typing
[20:52] <arigo> ?
[20:52] <arigo> ah I see
[20:52] <arigo> it would be nice actually if the cache files would all start with the same letter
[20:53] <arigo> so that we can rm _cache/c*
[20:53] <arigo> (if the letter is c)
[20:53] <arigo> as opposed to rm _cache/* which kills __init__.py
[20:54] <stakkars> sure, but you also can write _cache/_[tab key][enter]
[20:54] <stakkars> which executes __init__.py which kills all but itself
[20:54] <arigo> yes
[20:54] <stakkars> ok, the letter will me a for arigo :)
[20:54] <arigo> though the most expected behavior would probably be not to check _cache in at all and regenerate it completely, then we can simply trash _cache.
[20:55] <stakkars> rm -rf _ca[tab][enter] is more to type, or not
[20:57] <arigo> it's more a matter of expectation
[20:57] <arigo> removing the _cache removes the cache... makes immediate sense
[20:57] <stakkars> ok, making _cache completely trashable, then,and drop the common letter thing?
[20:58] <stakkars> side note: there is *some* cost, because I need to md5 every source code at runtime
[20:58] <arigo> yes
[20:58] <arigo> right
[20:59] <stakkars> ok, ready for your final decision, I'mleaving in 42 seconds
[20:59] <arigo> I'm for it :-)
[21:00] <stakkars> thanks bye c.u.tomorrow
[21:00] <arigo> md5'ing 10'000'000 characters takes no noticable time, so it's fine
[21:00] <arigo> see you !
[21:00] Action: stakkars hopes that this will be true for translated md5.py as well geeee :-D
[21:00] stakkars (~tismer@rosine120.inf.fu-berlin.de) left irc:
[21:07] fredrik (fredrik@c83-248-135-181.bredband.comhem.se) joined #pypy.
[21:08] <pedronis> arigo: mmh, intval gets attached to W_Root
[21:09] <pedronis> inttype.descr__getnewargs__
[21:09] <pedronis> seems a candidate cause for this
[21:10] <pedronis> the __getnewargs__ may need to become multimethods
[21:10] <pedronis> and we are still missing some typechecking for descriptors
[21:15] <arigo> see you !
[21:15] <arigo> oups sorry
[21:15] <arigo> wrong window
[21:16] <arigo> missing typechecks should create this effect, I guess
[21:16] <pedronis> yes
[21:16] <pedronis> the __getnewargs__ need to be multimethods likely
[21:17] <pedronis> and we need some more typechecking in other *type.py defined descr
[21:17] <arigo> er, I know next to nothing about __getnewargs__
[21:17] <arigo> I see
[21:17] <pedronis> it was added in Washington, it is for pickling
[21:17] <arigo> yes but here my knowledge stops :-)
[21:18] <pedronis> well, as usual it is a matter of thinking about supporting multiple different impl for the types
[21:18] <pedronis> in that case it make sense to have it a multimethods
[21:18] <pedronis> as a
[21:19] <arigo> ok
[21:19] <arigo> makes sense
[21:19] <pedronis> I will do that because is a bit less messy that doing typechecking in the *type.py files
[21:19] <pedronis> because of order issue
[21:19] <pedronis> I think we discussed this once
[21:20] <arigo> ah
[21:20] <pedronis> in inttype.py you cannot refer to W_IntObject
[21:20] <pedronis> because of the order in which things are defined
[21:21] <pedronis> intobject.py expects to be able to import inttype.py when define W_IntObject
[21:21] <arigo> yes
[21:21] <arigo> I see
[21:21] <pedronis> so you cannot use W_IntObject in the unwrap_spec in inttype
[21:22] <arigo> makes sense, yes.
[21:22] <pedronis> anyway having __getnewargs__ being a MM makes sort of sense
[21:22] Nick change: idnar_ -> idnar
[21:22] <pedronis> for other thing I fear I will need some kind of lazy specifier for unwrap_specs
[21:23] <pedronis> that will generate a local import in the unwrapping code
[21:23] <arigo> beuh
[21:23] <pedronis> well, typetype is already rolling is own version of that
[21:24] <pedronis> with _check
[21:24] <pedronis> unless we can solve the *type.py *object.py order
[21:25] <pedronis> we need something like that
[21:26] <arigo> well, but ultimately all these descriptors shouldn't be defined in typetype.py
[21:26] <pedronis> well, typetype is the exception
[21:26] <pedronis> because we said we don't care about multiple impl for it
[21:27] <pedronis> for types
[21:27] <arigo> ok ok
[21:28] <pedronis> I think once __getnewargs__ is a multimethod
[21:28] <pedronis> if have exceptions only in about type and object
[21:29] <pedronis> apart the descr__new__ that are special
[21:29] <pedronis> I have to go, see you
[21:29] pedronis (pedronis@ratthing-b246.strakt.com) left irc: "Chatzilla 0.9.67 [Firefox 1.0.2/20050325]"
[21:29] <arigo> raaaah I can't book my flight because the web site doesn't realize I select another flight than the first choice!!
[21:44] <arigo> oh dear
[21:44] <arigo> I think it's not by browser's fault, but then if it isn't it's the worst user interface I've ever seen
[21:45] <arigo> you have to click on some other button before you click on "Next" otherwise your choice changes are ignored.
[21:45] <arigo> makes perfect sense.
[21:48] <hpk> :-)
----- silence for 24 minutes ----- [22:12] <arigo> see you
[22:12] arigo (~arigo@vicky.ecs.soton.ac.uk) left irc: "later"
[22:25] pedronis (~Samuele_P@c-2b8b70d5.022-54-67626719.cust.bredbandsbolaget.se) joined #pypy.
----- silence for 1 hr and 23 minutes ----- [23:48] lac (jacob@c-51c6e055.1321-1-64736c11.cust.bredbandsbolaget.se) left #pypy.
[23:51] tea (~tea@cap058-158.kcn.ne.jp) joined #pypy.
[00:00] --- Thu Apr 21 2005