[00:54] arigo (~arigo@c-3a8b70d5.022-54-67626719.cust.bredbandsbolaget.se) left irc: Remote closed the connection
----- silence for 4 hr ----- [04:54] tea (~tea@cap010-137.kcn.ne.jp) joined #pypy.
----- silence for 25 minutes ----- [05:19] yuuh (hrpmgbf@i528C1E27.versanet.de) left irc: "utz utz utz"
----- silence for 2 hr and 11 minutes ----- [07:30] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) left irc:
[07:33] sanxiyn (tinuviel@211.104.100.177) joined #pypy.
----- silence for 21 minutes ----- [07:54] sanxiyn (tinuviel@211.104.100.177) left irc: "Bye"
----- silence for 1 hr and 11 minutes ----- [09:05] dialtone (~dialtone@212.97.38.145) joined #pypy.
[09:10] tea (~tea@cap010-137.kcn.ne.jp) left irc: Remote closed the connection
----- silence for 2 hr and 21 minutes ----- [11:31] hpk (~hpk@merlinux.de) left irc: Read error: 110 (Connection timed out)
[11:34] hpk (~hpk@merlinux.de) joined #pypy.
----- silence for 22 minutes ----- [11:56] arigo (~arigo@c-3a8b70d5.022-54-67626719.cust.bredbandsbolaget.se) joined #pypy.
[12:08] <hpk> arigo: morning, armin
[12:08] <arigo> morning
[12:09] <hpk> pestering you again: did you notice there was a greenlet question on py-dev?
[12:09] <arigo> a problem with py.test is that when a problem in PyPy finds a bug in py.test, I tend to fix the bug in PyPy first and then forget about the py.test one :-)
[12:09] <arigo> no, thanks
[12:11] <hpk> py.test is not robust and clear enough with respect to representing things
[12:16] <arigo> hpk: thanks for pointing out the greenlet question, I guess it was amonst the mails I lost.
[12:16] yuuh (hkwihif@i528C1E27.versanet.de) joined #pypy.
[12:16] <hpk> i guess so
----- silence for 24 minutes ----- [12:40] <hpk> arigo: i just sped up PyPy's testing process by 10% by refactoring py.test a bit :-)
[12:40] <arigo> :-)
[12:49] <hpk> would it be helpful to have per-test hotshot anaylsis, btw?
[12:49] <arigo> hard to guess
[12:51] <arigo> in general, it would probably be helpful; for PyPy, no idea, we need to try hotshot on some examples
[12:51] <hpk> for optimizing standardobjectspace it was quite helpful i remember
[12:52] <arigo> from my point of view there are lots of other things to do on PyPy at the moment...
[12:52] <arigo> giving it a quick try might be useful, I just have no clue
[12:53] <hpk> sure
[12:58] <hpk> i've yet to find an area in translation that i can/want to work on
[12:58] <hpk> arigo: do you recommendations?
[12:59] dialtone (~dialtone@212.97.38.145) left irc:
[13:04] <hpk> some of your recent translator checkins look quite mysterious
[13:07] <arigo> translator, or annotator?
[13:10] <hpk> annotator i guess
[13:11] <arigo> well, yes, that was some tweaking for obscure cases we encounter in PyPy
[13:11] <arigo> the annotator still works quite nicely, it's not where more work is needed now
[13:12] <hpk> i guess i am quite used to having a workstyle where i can test things rather that just fix random places without obvious reasons
[13:12] <hpk> arigo: why were you asking then? :-)
[13:13] <arigo> well, all these changes have reasons, obviously :-)
[13:14] <arigo> but yesterday's work was really only cleaning up a few remaining loose ends from Samuele's previous great work,
[13:14] <arigo> for which he wrote lots of tests
[13:14] <arigo> the places where work is needed in PyPy are fixing things discovered by the CPython tests, and C translation.
[13:18] <hpk> is it still the case that genc doesn't use annotation info?
[13:18] <arigo> no, it makes some use of it
[13:19] <arigo> there is the basic CTyper, which is probably good already
[13:19] <arigo> in translator/typer.py
[13:20] <arigo> the general organization of translator/genc/* might need changes again,
[13:20] <arigo> the documentation I wrote is partially a wish list.
[13:24] stakkars (~tismer@rosine133.inf.fu-berlin.de) joined #pypy.
[13:30] yuuh (hkwihif@i528C1E27.versanet.de) left irc: Read error: 110 (Connection timed out)
[13:30] Action: hpk -> breakfastlunch
[13:30] yuuh (olmvqf@i3ED6B5C2.versanet.de) joined #pypy.
----- silence for 32 minutes ----- [14:02] bd (~bea@3-1-8-23a.gva.gbg.bostream.se) joined #pypy.
[14:02] <bd> hi - hpk - you there?
[14:11] bd (~bea@3-1-8-23a.gva.gbg.bostream.se) left irc:
[14:23] <arigo> hi all
[14:24] <stakkars> hi
[14:26] <arigo> stakkars: I thought about moving test_typer out of the way while it was failing, but then I noticed you were working on it today, please ignore my latest two check-ins
[14:26] dialtone (~dialtone@host244-1.pool21345.interbusiness.it) joined #pypy.
[14:27] <stakkars> I should have checked in more consistently.
[14:28] <stakkars> in a way I'm in a hurry to change transform back after the changes to simplify are done,
[14:28] <stakkars> but I wanted to increase the checkin gap again.
[14:28] <stakkars> s/wanted/wanted to avoid to/
[14:30] <stakkars> ha! Here the bug with rpystone
[14:30] <stakkars> run translate_pypy targetrpystone three times.
[14:31] <stakkars> the third time will give a segfault.
[14:32] dialtone (~dialtone@host244-1.pool21345.interbusiness.it) left irc:
[14:32] dialtone (~dialtone@host244-1.pool21345.interbusiness.it) joined #pypy.
[14:34] <arigo> I see
[14:35] <stakkars> does it crash for you, too?
[14:35] <stakkars> it is flaky
[14:37] <arigo> indeed
[14:37] <arigo> I got a crash the 2nd time, not the 1st one
[14:38] <stakkars> and if you look into the source code, it is really wrong code. some gint_5 doesn't get generated, instead it's a native one.
[14:38] pedronis (pedronis@ratthing-b246.strakt.com) joined #pypy.
[14:39] <arigo> stakkars: is that really wrong?
[14:40] <arigo> stakkars: yes, sorry
[14:40] <arigo> stakkars: no!
[14:40] <arigo> it's not a native one, it's in the .py file
[14:40] <arigo> which is interpreted at module load-time
[14:41] <arigo> so you get a PyIntObject
[14:43] <stakkars> line 1683 reads like this:
[14:44] <stakkars> OP_SETATTR(ginst_G, gstr_IntGlob, 5, v49, err4_23)
[14:44] <arigo> oups, a 5 :-)
[14:44] <stakkars> which gives me a compiler warning and of course a crash.
[14:44] <arigo> that's a problem in the CTyper
[14:45] <stakkars> and note that this happens some times and sometimes not!
[14:45] <arigo> I see
[14:45] <stakkars> how is the typer dependent of the status of CPython?
[14:45] <stakkars> the only difference is existence of .pyc files :-(
[14:45] <stakkars> I'vespent hours on that with no result, yet.
[14:47] <arigo> if a constant gets written out as "5", it means it thinks it's a CIntType
[14:47] <arigo> ack! windows end-of-line in ctyper.py!
[14:47] <arigo> :-)
[14:48] <arigo> stakkars: the problem is probably in instancetype.py, spec_setattr()
[14:50] <arigo> stakkars: ah, maybe not
[14:50] Action: arigo investigates another hypothesis
[14:51] Action: stakkars has the hypothesis that we have a serious python bug, here
[14:51] <arigo> no
[14:52] <arigo> I know what it is
[14:52] <arigo> if you go to the rpystone.py source, Proc8,
[14:52] <stakkars> yes
[14:52] <arigo> and replace g.IntGlob = 5 with g.IntGlob = -2345
[14:52] <stakkars> yes
[14:52] <arigo> then it doesn't segfault any more
[14:52] <stakkars> ouups?
[14:53] <arigo> the reason, as I guess, is that in some cases this '5' is the same Constant object as the '5' above in Proc8
[14:53] <arigo> and in other cases (depending on unreproductible details) it's another Constant object
[14:54] <stakkars> which 5do you mean?
[14:54] <arigo> in the first line
[14:54] <arigo> IntLoc = IntParI1 + 5
[14:55] <stakkars> ah,oh. But still I don't get why in some cases it works.
[14:55] <arigo> both '5' are Constant objects, but sometimes two different ones and sometimes the same one
[14:55] <arigo> if they are the same one, trouble:
[14:55] <stakkars> sure
[14:55] <arigo> the ctyper will attach a 'concrete_type=TInt' attribute to the one in the first line
[14:56] <arigo> this will accidentally make the '5' in the last line also an int, instead of a PyObject
[14:56] <arigo> a bad idea of mine, to attach the concrete_type on Constants just like that
[14:56] <stakkars> ah. The constant is a singleton. hee hee. Constant(5) will be the same.
[14:56] <stakkars> holds for all small numbers
[14:57] <arigo> no, nothing reproducible
[14:57] jacob22|home (~jacob@c-51c6e055.1321-1-64736c11.cust.bredbandsbolaget.se) joined #pypy.
[14:57] <stakkars> I see, the constant objects should be different
[14:58] dialtone (~dialtone@host244-1.pool21345.interbusiness.it) left irc:
[14:58] <arigo> in principle, the flow objspace creates a new Constant instance per call to wrap(), so they should be different
[14:58] lac (~lac@lac.silver.supporter.pdpc) joined #pypy.
[14:58] <arigo> I guess there is some obscure reason that they might become the same
[14:58] <lac> hi all
[14:58] <pedronis> lac: hi
[14:58] <arigo> so the real bug is in ctyper, which should not attach concrete_type to existing Constants like that
[14:58] <arigo> hi!
[15:00] <stakkars> we have a nice bug in app_complex, line 108. "inf" is platform specific.
[15:00] <stakkars> and does not work on winnows.
[15:03] <lac> so we have a meeting here? jacob is back from England and in the kitchen now :-)
[15:04] <pedronis> lac: the meeting is on #pypy-funding
[15:04] <lac> aha
[15:04] <lac> thanks samuele.
----- silence for 1 hr and 4 minutes ----- [16:08] dialtone (~dialtone@host51-0.pool21345.interbusiness.it) joined #pypy.
[16:10] dialtone (~dialtone@host51-0.pool21345.interbusiness.it) left irc: Remote closed the connection
[16:11] dialtone (~dialtone@host51-0.pool21345.interbusiness.it) joined #pypy.
----- silence for 1 hr and 2 minutes ----- [17:13] lac (lac@lac.silver.supporter.pdpc) left #pypy.
[17:16] dialtone (~dialtone@host51-0.pool21345.interbusiness.it) left irc:
[17:24] <stakkars> hpk, arigo: about your conversation from yesterday: we can add targetrpystone and targetpypy as regular tests now
[17:34] <arigo> yes, and targetpypymain appears to work nicely again.
[17:34] <arigo> not "again", even: for the first time -- it's a few one :-)
[17:35] <arigo> s/few/new
[17:39] <stakkars> arigo: I did a rpystone crash, again, then I removed all .pyc files and started over.
[17:39] <stakkars> what I get is really different C code.
[17:39] <stakkars> with type being wrong or not, how can you explain that?
[17:39] <stakkars> (or better to say, we should explain it if we want to trust our code base :-) )
[17:40] <stakkars> s/type/typer/
[17:40] <stakkars> see tismerysoft /tmp/usession38 vs 39
[17:43] <arigo> the diff is large, can you point me at something that is not just from some different dict key order?
[17:43] Action: stakkars sees arigo diffing :-)
[17:44] <arigo> ok, I found the OP_SETATTR with once a 5 and once a gint_5, but that's what I said earlier
[17:45] <stakkars> I was not sure if it was source code difference or just a side effect of the memory.
[17:47] <stakkars> oops, targetmain crashes in my hacky code
[17:47] <stakkars> and my machine swapps like hell
[17:47] <stakkars> again, my problem is how can we explain non-deterministic behavior, if this is the same program and data?
[17:48] <arigo> very easy in Python
[17:49] <arigo> dict key order being the primary explanation
[17:49] <stakkars> ah, ok. That's fine of course
[17:50] <arigo> I'm not entierely sure about what occurs in this case, though
[17:50] <stakkars> maybe we don't want this. Wherever I can remove entropy I do it
[17:51] <arigo> don't go changing genc all around for now,
[17:51] <pedronis> stakkars: we have discussed some possibilities
[17:51] <arigo> but yes for the annotator for example it is a meaningful thing to do
[17:52] <pedronis> like having a total order for blocks
[17:53] <stakkars> yes, having pipe-like behavior. Also, log files of the annotation might become diffable
[17:53] <pedronis> we should try, it is not completely clear if there is more that would not be covered by that
[17:53] <stakkars> is there anything else but dicts with nondeterministic enumeration order?
[17:54] <stakkars> well, using id(object) may cause entropy, too
[17:55] <pedronis> no, we thought of using some hashing of invariant stuff in code objects and bytecode positions
[17:55] <pedronis> things that are ordered by id are likely the cause of the non-determinism in the first place
[17:56] <stakkars> ah, yes.
[17:56] hpk (~hpk@merlinux.de) left irc: "Client Exiting"
[18:00] <arigo> stakkars: I fixed the problem by never adding a concretetype attribute to an old Constant instance.
[18:00] <arigo> so yes, it was a real bug
[18:01] <arigo> btw several literals '5' in the same function use the same Constant because co.co_consts is wrapped only once
[18:06] <stakkars> great!
[18:08] <stakkars> works fine. Is pypymain supposed to work, too?
[18:09] <arigo> the annotation phase, yes
[18:09] <arigo> yesterday it crashed in transform_ovfcheck(), so i don't know
[18:09] <stakkars> ok, then I caused something.
[18:10] Action: arigo tries again
[18:10] <stakkars> don't try
[18:11] <stakkars> I am fixing things
[18:11] <arigo> ok :-)
[18:12] <stakkars> and I am thinking about doing more on exceptions, like completely tracking what a function can raise.
[18:12] <stakkars> Then we can prune the tree for many simple_calls, too. (dynamic can_only_throw)
[18:13] <arigo> don't forget that the annotator is more difficult to understand, but more powerful in doing these things, too
[18:13] <stakkars> I meant the annotator, yes.
[18:13] <arigo> ah, sorry, I was looking at transform.py.
[18:14] <stakkars> we can add a phase [hey, I wrote in the check-in message that you *may* not look]
[18:14] <stakkars> that's only there because I had no time to get things finalized
[18:15] <arigo> oups :-) I just looked enough to see that you're not using the 'self' argument, a sure sign that the same code can be moved to simplify.py
[18:15] <arigo> ('self' = annotator)
[18:15] <arigo> but you know that :-)
[18:15] <stakkars> that code will go nowhere because it is very bad.
[18:15] <arigo> :-)
[18:16] <arigo> btw, a quick check-in that comments out the call to transform_ovfcheck() is always appreciated by people working on targetpypymain :-)
[18:17] <stakkars> does it work without that, then?
[18:17] <stakkars> ok, I'll do
[18:17] <arigo> ah, right, maybe not
[18:17] <stakkars> that was why I left it in :-/
[18:17] <arigo> that was the whole point, the other translatepypy* don't work without that either :-\
[18:18] <arigo> sorry
[18:18] <stakkars> yes, I was too far with other stuff, doesn't work any longer if you don't rewrite
[18:19] <stakkars> anyway, the idea is to carry extra info for every procedure about possible exceptions. Finally,
[18:19] <stakkars> all simple_call instances can be treated like the implicit stuff.
[18:25] <arigo> it looks like this information should appear as an annotation in the graph.exceptblock's input args,
[18:26] <arigo> like the return value, which is the annotation of the graph.returnblock's input arg...
[18:26] <pedronis> yes, we may need to think again about how to make more precise what enters there
[18:27] <pedronis> becasue sometimes you get SomeObjects
[18:27] <pedronis> especially in cases were there are explicit raises or re-raises
[18:27] <pedronis> and for this these are cases you really want the info
[18:28] <arigo> stakkars: in this point of view, the problem consists in designing SomeXxx correctly, and putting them to the correct variables when we see an exception handler in function. Am I making sense?
[18:30] <stakkars> probably. Exceptions appear to be similar to normal exits and can described that way.
[18:31] <arigo> yes.
[18:31] <arigo> in other words, in flowin(), attach precise annotations; let them flow forward by themselves -- they need a careful union() for that; and eventually look at what shows up on the exceptional exit block.
[18:31] <stakkars> in that sense, our analysis isn't complete, yet.
[18:31] <pedronis> yes, but may want to keep a set of separate classes instead of using the usual SomeInstance union
[18:31] <pedronis> for the except block
[18:32] <arigo> well, it's imprecise, in this respect.
[18:32] <pedronis> yes
[18:32] <pedronis> and there are other subtle points for sure
[18:32] <arigo> I'm unclear if this is essential missing precision or not...
[18:33] <pedronis> well, it depends
[18:33] <pedronis> once you get more than one Exception subclass
[18:33] <pedronis> your type will be Exception
[18:34] <pedronis> and there are probably other things at play too
[18:34] <pedronis> for example the type vs the value
[18:35] <pedronis> easely degenerate to SomeObject
[18:35] <pedronis> etc
[18:35] <arigo> what I was wondering is
[18:35] <arigo> now that we have figured out that we can pass a large number of CPython tests with a few days' hacking efforts, the big open question is if we can compile to reasonable C speed :-)
[18:35] <arigo> however it's probably fine to keep doing things in parallel.
[18:36] <stakkars> if we can remove 70 % of all exception cases, we will save over 50% of the code, and know the answer much quicker :-)
[18:36] <arigo> I see
[18:36] <pedronis> I'm just saying that it needs to be tried and depending how much precision we want
[18:37] <arigo> stakkars: but I'd love to involve you now in dirty details of C code generation :-)
[18:37] <pedronis> we may need to rework how exception handling appears in flow graphs
[18:37] <stakkars> arigo: sure
[18:37] <pedronis> because after some point getting precision in the annotator given how exc handling looks now gets very messy
[18:38] <stakkars> btw. I don't want to do so much, just find out which exceptions the function can emit.
[18:39] <stakkars> That can be done in a quite systematic way, starting with those functions which don't call any functions.
[18:39] <stakkars> For them, we have complete info.
[18:39] <arigo> well
[18:39] <arigo> that's precisely the direction in which I'd like to avoid to go
[18:39] <stakkars> Then this becomes an atrrribute of the function somehow, and we proceed to the callers.
[18:39] <stakkars> too simple? :-)
[18:39] <arigo> the annotator doesn't work callee-first
[18:40] <stakkars> that's no problem.
[18:40] <arigo> it seems to me that what you describe can only work as a post-processing phase, and then all samuele and me said above -- annotations, flow them -- is irrelevant
[18:40] <pedronis> you can do such an approach after the fact but you don't gain precision and
[18:41] <stakkars> I am talking of the logical process that I wantto take place, not the order we are actuallydoing it.
[18:41] <pedronis> you are on your own doing the pruning and dealing with OO call sites
[18:41] <pedronis> stakkars: arigo just pointed out that you can get the result quite naturally out of what the annotator already does
[18:42] <pedronis> I just reminded (because I worked quite a bit on that) that augmenting
[18:42] <stakkars> simply walking the graphs and collecting.
[18:42] <pedronis> the precision of what enters exception blocks
[18:42] <pedronis> needs more work
[18:43] <pedronis> and it may be usable in general
[18:43] <pedronis> because it reduce the amount of spurious SomeObject floating around
[18:43] <stakkars> ok. I was just after making everything smaller, having less blocks floating around.
[18:43] <pedronis> in exc handling code
[18:44] <arigo> stakkars: the fact is that a nicely integrated and more general solution is possible with the annotator itself, but needs more work
[18:44] <pedronis> but my impression that given how exc handling code looks like now it not totally straightforward without avoiding even more strange special casing
[18:44] <arigo> more general in the sense that it won't depend on syntactic details on how the blocks look like, e.g. in the case of re-raises, etc
[18:44] <pedronis> so we may need to rework how exc handling look in the graphs in the first place
[18:45] <stakkars> we could surely find a better representation. Right now we have the long traces of the interpreter trying to find out.
[18:48] <arigo> I promized to start a pypy-dev discussion about genc... me culpa.
[18:49] <stakkars> I shut up, then :)
[18:49] <arigo> another point, though: the annotator is expected to be extendable to work like Psyco, at run-time
[18:49] <arigo> at least to some extent
[18:50] <arigo> but the global analysis phase you describe would be more difficult to do "just-in-time"
[18:51] <stakkars> I did not describe its physical flow.
[18:51] <arigo> well, it's delicate:
[18:51] <stakkars> I would just try to do this stuff during flow analysis.
[18:51] <arigo> if you're doing things just-in-time, you are never sure that you've seen all possible exceptions that a function can raise. more may show up later
[18:52] <pedronis> yes, in the annotator the set of possibly raised exception would grow over time
[18:52] <pedronis> and you would analyses more handling code
[18:53] <stakkars> right. But I would be sure if I know that a function is fully annotated.
[18:53] <arigo> no: that's the kind of assumption that makes what you describe incompatible with running inside the annotator :-)
[18:54] <arigo> "fully annotated" has no meaning, basically, until the annotator globally finished
[18:54] <arigo> so you're describing a post-processing phase.
[18:55] <arigo> you never know if a given block could be reflown later or not...
[18:55] <pedronis> well, you would have similar effects to what going from False|True to SomeBool has
[18:56] <arigo> ?
[18:57] <pedronis> if you have conditional code like
[18:57] <pedronis> if x == "a":
[18:57] <pedronis> ...
[18:57] <pedronis> elif x == "b":
[18:57] <pedronis> ...
[18:57] <pedronis> and you start with x constant = "a"
[18:57] <pedronis> you first ignore the second block
[18:57] <pedronis> until x is generalized
[18:58] <arigo> yes, and the second block might be 'raise ANewException'
[18:58] <arigo> but I think Christian wants to look at the flow graphs only to know what exceptions can be thrown
[18:59] <pedronis> no what I'm saying is that you would have similar behavior with the respects of the exception exitcases
[18:59] <pedronis> the set of exceptions would grow and you would so consider more exitcases
[18:59] <arigo> yes
[19:01] <pedronis> so yes dynamically you have no exact information about the ignorable exceptions
[19:01] <pedronis> but is not that much different from not being sure that will not enter the elif x=="b" block above
[19:04] <stakkars> hmm.
[19:08] <stakkars> so I would start off with the assumption that there are no exceptions possible, until I know better. hmm.
[19:09] <stakkars> that raises the question: How do you make sure that you really get all the type assumtions right?
[19:09] <pedronis> well, when the analysis is static when you reach steady-state
[19:10] <pedronis> if you are doing thing dynamically you should always have fallbacks for when the assumptions reveal themself as wrong
[19:10] <arigo> before that, there is dependency tracking, in the form of "when this thing here is found to be more general that previously thought, restart annotation of that other block over there"
[19:11] <arigo> for example, the return value of a function, which when generalized triggers reflowing of all the blocks that called the function (these blocks usually belonging to other functions, of course)
[19:11] <pedronis> yes
[19:13] <stakkars> ok. And you would try to turn exceptions into other return values with the same effect.
[19:14] <arigo> yes
[19:14] <pedronis> I think you would have similar treatement for except block as we do for return blocks
[19:14] <arigo> when a new exception escapes the function, it would trigger the invalidation of all the blocks that depended on the previous known can_throw list.
[19:15] <arigo> yes
[19:16] <stakkars> seems to be the right way. As you said, not trivial like my approach that would be quite simple but less accurate.
[19:19] <stakkars> hmm, I commented my ovf transform completely out, but still get crashes on targetmain
[19:19] <pedronis> FIY: I have checked in a modified test_descr, so that we don't die so abrutly on failures (at least if is not PyPy "segfaulting") with results
[19:20] <stakkars> I|m no longer reallz convinced that this is on mz side
[19:21] <stakkars> please read with german keyboard overlay on american keyboard, I hit a key that switched me :-)
[19:22] <arigo> ö'(
[19:22] <arigo> :-) with swiss/us confusion
[19:29] <stakkars> waah. as always it works on trivial examples, but targetmain just crashes with assert len(argtypes) == len(args)
[19:33] <stakkars> can one of you please try? I did so much debugging the last says, yeeech
[19:33] <arigo> is that with no particular option?
[19:33] <stakkars> just -text
[19:33] <arigo> I just tried with -no-t, it dies while generating C code
[19:34] <arigo> I'm not sure I trust the CTyper for targetpypymain -- in fact I'm sure I don't :-) but let me try...
[19:34] <stakkars> saying what?
[19:34] <arigo> no typer
[19:34] <stakkars> no, with which death message?
[19:34] <pedronis> I think I was getting that assertion also with targetpypy
[19:35] <arigo> <built-in function backslashreplace_errors> not found in any built-in module
[19:35] <stakkars> that's a new one.
[19:35] <pedronis> I saw that one too
[19:40] <arigo> probably stringobject.py which contains applevel-ifed 'import codecs'
[19:41] <arigo> stakkars: the assertion error in the typer:
[19:41] <stakkars> yes
[19:41] <arigo> it's a function call that doesn't provide enough argument, because there are default args
[19:42] <arigo> I guess I forgot about default args when I wrote this code :-)
[19:42] <stakkars> :-) easy to fix? I mean I smode a cig and continue testing? :-)
[19:43] <arigo> well, let's try to figure out how to generate C code for default args...
[19:43] <arigo> given that the function we call might be a function pointer.
[19:43] <stakkars> one extra function for every arity
[19:44] <stakkars> would be simple I guess?
[19:44] <pedronis> well, the annotator already does that kind of specialisation
[19:44] <pedronis> but is not visible in the graphs
[19:45] <stakkars> you mean it should not slip through and generate specialized headers?
[19:45] <arigo> wait, there is some missing piece
[19:45] <arigo> let's take an exemple of a call f(x,y) where f is a variable
[19:46] <arigo> do we allow f to point to any function that accepts two arguments of the correct type?
[19:46] <pedronis> I think that's not supported
[19:46] <arigo> what is not supported? just f(x,y)?
[19:46] <pedronis> well, you should look at what happens in bookkeeper
[19:47] <arigo> but bookkeeper is called for one concrete function
[19:47] <pedronis> on one hand a site accepts only functions
[19:47] <arigo> several times if needed
[19:47] <pedronis> with similar signatures
[19:47] <pedronis> OTOH bookkeeper
[19:47] <pedronis> for a function with default arguments creates a family of functions
[19:47] <pedronis> f_#
[19:47] <pedronis> if I recall
[19:48] <pedronis> but I think we need check how this two things interact
[19:48] Action: arigo confused
[19:49] <pedronis> I think that a function f with default arguments matches in recursivecall
[19:49] <pedronis> but then bookkeeper will consider the call as a call to the right element in the family
[19:49] <arigo> wait,
[19:49] <arigo> def g(x): return x+1
[19:49] <arigo> def h(x, n=2): return x+n
[19:49] <arigo> def fn(x):
[19:49] <arigo> if x:
[19:49] <arigo> myfunc = g
[19:49] <arigo> else:
[19:49] <arigo> myfunc = h
[19:49] <arigo> return myfunc(x)
[19:49] <arigo> this produces the correct annotations.
[19:50] <pedronis> I think so
[19:50] <arigo> i.e. no warning, and fn(int) returns an int.
[19:50] <pedronis> yes but as I said
[19:50] <arigo> I'm not discussing the annotator, I'm discussing how to translate this to C
[19:50] <pedronis> I think that at the call site
[19:50] <pedronis> the annotator is considering g, h_1
[19:50] <pedronis> g and h_1
[19:51] <pedronis> that means that type of myfunc should be a PBC with those elements
[19:51] <arigo> but if I add a call h(5,7) somewhere else, things are still correct
[19:51] <arigo> i.e. a single h function
[19:52] <pedronis> yes
[19:52] <pedronis> but
[19:52] <pedronis> myfunc will be the PBC g,h
[19:52] <pedronis> but internally the annotator is really considering
[19:52] <pedronis> g,h_1
[19:52] <arigo> ok let me explain with another example...
[19:53] <arigo> I'm asking how, when the translator sees the line
[19:53] <arigo> myfunc = h
[19:53] <arigo> hum, no wait
[19:53] <pedronis> well, the problem
[19:54] <pedronis> is that you could have g also have a default arg
[19:54] <pedronis> and then call myfunc
[19:54] <pedronis> with one or two arg
[19:54] <arigo> yes
[19:54] <pedronis> the annotator is happy
[19:54] <arigo> yes, that's the kind of example I was looking for
[19:54] <pedronis> but that doesn't mean that generating code
[19:54] <pedronis> is that straightforward
[19:55] <pedronis> the problem is the order
[19:55] <arigo> yes I'm asking how to generate code for this kind of example
[19:55] <pedronis> in which arg matching is done
[19:55] <pedronis> vs. specializing
[19:55] <arigo> in this case specializing is an internal detail of the annotator
[19:56] <pedronis> yes
[19:56] <arigo> when you pass a constant 'g' around, you don't know with how many arguments it will eventually be called. What should the C variables containing 'g' be?
[19:56] <pedronis> well as I said you can see it as problem
[19:57] <pedronis> that the annotator is happy
[19:57] <arigo> I wasn't really assuming that, but more that we can find a solution
[19:57] <arigo> for the C side
[19:58] <arigo> I was really asking a C-hacking question, expecting answers like "well, pass around a struct with several pointers for the various arities"
[19:58] <stakkars> for the moment, maybe we could just avoid the situation?
[19:58] <pedronis> well, unless you can find out a way to track more about how myfunc={g,h} is used
[19:58] <pedronis> yes you need a table function * arity -> function
[19:58] <stakkars> why do we use such constructs in the interpreter, is it needed?
[19:59] <arigo> I don't know exactly
[19:59] <pedronis> probably not
[19:59] <stakkars> maybe we just identified an unrpythonic spot
[19:59] <arigo> in this case it's just a plain call to a single function expecting default args
[20:00] <arigo> so it's easy to fix
[20:00] <pedronis> yes
[20:00] <pedronis> you can resolve it at generation time
[20:00] <arigo> we can also say that default args are part of the signature,
[20:01] <arigo> i.e. if you call a variable pointer, all possible functions in the set must require the same extra default args to be inserted
[20:01] <arigo> then the C typer knows which additional arguments to supply.
[20:02] <pedronis> yes, but now I'm confused maybe the annotator does things in the reverse order
[20:02] <stakkars> I think that would be fine, and even better if we can detect violations of that.
[20:02] <arigo> stakkars: yes, that should be easy
[20:03] <arigo> stakkars: because we know the set of concrete functions that a given variable can take, actually
[20:03] <stakkars> btw. I would go further and forbid default args on function pointers.
[20:04] <arigo> well, there is not real difference between functions and function pointers...
[20:04] <arigo> apart from "being only allowed to point to a single function" or not
[20:04] <arigo> but yes, it makes some sense, I guess
[20:06] <stakkars> even further, I'd like to completely get rid of any default stuff in RPython
[20:06] <arigo> why?
[20:06] <pedronis> no, sorry, it's all done by the bookkeeper, recursivecall isn't dealing with this
[20:07] <arigo> stakkars: it's not like they are a major burden, and they are a major help :-)
[20:07] <pedronis> so it's just a matter of dividing tasks betweeen pycall and its caller
[20:07] <pedronis> callers
[20:07] <arigo> pedronis: I don't see the point of changing the annotator for that
[20:07] <stakkars> if we need them much, fine. If it is for sloppy coding, throw it away.
[20:08] <pedronis> arigo: if you want the check
[20:08] <pedronis> of course you could check at generation time too
[20:09] <arigo> you'll have to, basically
[20:09] <arigo> at least as an assert
[20:09] <arigo> if we're sure we reproduce the exact same restriction in the annotator, maybe, but I don't see the point much
[20:09] <pedronis> well but you would spot problem earlier if it's done at annotation time
[20:10] <pedronis> it's true that some generators could be simply happy with default arguments
[20:10] <arigo> yes, as I said, this wasn't really a question about the annotator :-)
[20:10] <stakkars> pedronis: please, yes! crash as early as possible, especially for target main :-)
[20:12] <arigo> I don't expect this case to show up often, just asking what to do with it in GenC when it does :-)
[20:13] <arigo> for debugging, it would be much more profitable to think a bit about saving some of the state produced by the annotator
[20:17] <stakkars> you mean I continue with the pickling idea
[20:18] <stakkars> sure, we get that.
[20:18] <stakkars> Have to go home, I'mvery tired
[20:20] <arigo> I think pickling is potentially easier after annotator
[20:20] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) joined #pypy.
[20:20] <arigo> because we should only have "simple" constants left, ideally
[20:20] <arigo> and function objects, but these can be replaced by stubs
[20:21] <stakkars> yes. I will make sure that we can save state exactly before any genXXX will start.
[20:22] <stakkars> then, different back-ends can work on that and get compared, etc.
[20:22] <arigo> cool
[20:22] <stakkars> we pipe an annotator state over the network and c-compile there...
[20:22] <arigo> :-)
[20:24] <stakkars> at some time, I'd like to do annotation on geninterp, too. But that's a different story.
[20:24] <stakkars> bye now, have a nice evening
[20:25] <stakkars> wait,do you have a fix for that problem already? That I'd wait for the check-in
[20:27] <arigo> the default args? no, but it's likely that a quick fix would only cover 99% of the cases and then crash again
[20:28] <arigo> ah yes, I see a quick fix:
[20:28] <arigo> fall back to simple_call instead of using direct_call if something's strange about the # of args
[20:28] <stakkars> I just wanted to see if the fat thing compiles, it might show problems that trivial rpystone-alikes don't show
[20:29] <arigo> will probably crash on that same backslashreverse built-in function
[20:30] <stakkars> well, that's a matter of adding to the file I forgot which one, but I needed todo that for rpystone, too
[20:30] <pedronis> stakkars: annotating geninterp code would need a slightly different set of types
[20:30] <pedronis> and there may be other problems
[20:30] <arigo> (checked in)
[20:30] <stakkars> great!
[20:31] <stakkars> pedronis:yes. I was thinking of _formatting etc.,
[20:31] <stakkars> where I'm not sure how far we can break things down without prior annotation.
[20:32] <pedronis> I see, but I suspect doing enabling annotation is at that level is quite a bit of work
[20:33] <stakkars> maybe I'll do some tests at some time.
[20:34] <stakkars> last Q before I go:
[20:34] <stakkars> to make things running with Exceptions, I had to make sure that the space instance gets filled
[20:34] <stakkars> with attributes. You might have seen my check.in comments on that.
[20:35] <stakkars> I was then wondering why we produce the space object at all. Isn't it meant to be a constant?
[20:35] <stakkars> (this is the Q)
[20:37] <pedronis> I see
[20:38] <pedronis> well what you get are a lots of getattr(space,....) -> somepbc
[20:38] <pedronis> in some cases you could ignore the getattr and just consider the result
[20:39] <stakkars> yes. I had that with all the w_Exceptions
[20:40] <pedronis> the flowspace produce the getattr because the space is a Variable for it
[20:40] <pedronis> the annotator does not remove operations in itself
[20:41] <stakkars> is it our intent to keep it, or would you consider it a matter of further optimization?
[20:42] <pedronis> well it depends, it just that given the layering of things you don't get it for free
[20:42] <pedronis> successive phases should decide what they want to do
[20:43] Action: pedronis is about to leave the office
[20:43] <stakkars> see you
[20:43] <pedronis> see you
[20:44] pedronis (pedronis@ratthing-b246.strakt.com) left irc: "Chatzilla 0.9.67 [Firefox 1.0.2/20050325]"
[20:47] <arigo> stakkars: (sorry late) it's a job for CTyper typically to consider how to handle pbcs,
[20:48] <arigo> and a getattr on them is typically turned into nothing (in the same way as add on its is turned into add_int).
[20:48] <stakkars> ah you say we can decide it on that level.
[20:48] <stakkars> I just wanted to ask whether we have an intent to keepsomething like space around.
[20:49] <arigo> no,
[20:49] <arigo> it will disappear, probably along with all other SomePBC({...1 instance...})
[20:51] <stakkars> completely OT, but I'd like to make a different impl of space.is_w
[20:51] <stakkars> I have now all my operators in target1 working nicely, most of the time they pulled the
[20:51] <stakkars> interpreter in by asking for is_true(is_(something, other))
[20:52] <stakkars> where the is_ is just fine,
[20:52] <stakkars> but the is_true is really bad, because it tries to find the truth of a general object, involving
[20:52] <stakkars> descriptors.
[20:52] <stakkars> So to keepsmall examples manageable, it might make sense to directly implement is_w
[20:53] <stakkars> I did that for certain types just by replacing
[20:54] <stakkars> if space.is_true(space.is_(space.type(w_obj, W_Integer)))
[20:54] <stakkars> (or something)
[20:54] <stakkars> by if space.w_True is space.is_(....)
[20:55] <stakkars> eof for today, bye :)
[20:56] <arigo> yes makes ense :-)
[20:56] <arigo> bye
[20:57] <arigo> (would increase py.py performance a little bit too :-)
[20:58] <stakkars> well and I thought if it is possible to cut dependencies, it makes sense.
[20:58] <stakkars> bye
[20:58] <arigo> good night
----- silence for 19 minutes ----- [21:17] stakkars (~tismer@rosine133.inf.fu-berlin.de) left irc: Read error: 113 (No route to host)
----- silence for 41 minutes ----- [21:58] yuuh (olmvqf@i3ED6B5C2.versanet.de) left irc: "utz utz utz"
----- silence for 1 hr and 13 minutes ----- [23:11] arigo (~arigo@c-3a8b70d5.022-54-67626719.cust.bredbandsbolaget.se) left irc: Read error: 104 (Connection reset by peer)
[23:11] arigo_ (~arigo@c-3a8b70d5.022-54-67626719.cust.bredbandsbolaget.se) joined #pypy.
[23:12] Nick change: arigo_ -> arigo
----- silence for 43 minutes ----- [23:55] aleale (~root@cpe.atm0-0-0-129140.0x3ef2fa3a.bynxx3.customer.tele.dk) joined #pypy.
[00:00] --- Sat May 7 2005