[00:06] hpk (~hpk@merlinux.de) joined #pypy.
[00:07] <hpk> evening
----- silence for 16 minutes ----- [00:23] <arigo> evening
[00:24] <hpk> can we come up with worthwhile translation tests to run repeatedly?
[00:24] <hpk> i mean from the goal directly?
[00:24] <hpk> directory even
----- silence for 21 minutes ----- [00:45] lac (~lac@lac.silver.supporter.pdpc) left irc: "using sirc version 2.211+KSIRC/1.3.11"
[00:51] <arigo> hpk: not right now
[00:52] <arigo> small changes tend to break everything there
[00:52] <hpk> yes, as is currently the case
[00:52] <arigo> and I'm not helping with changing small and essential dark corners
[00:52] <hpk> but then it would be interesting to track down exactly which checkin broke things
[00:52] <arigo> lots of them did recently :-)
[00:53] <arigo> Samuele and me are trying to get things back to shape
[00:53] <arigo> in addition, just looking at the output is not useful to understand anything
[00:54] <arigo> you really need to walk around in the graph and/or inspect in the debugger
[00:55] <arigo> well maybe we can run the goals daily and record the results, but only some things are easily tracked through revisions that way
[01:03] Action: hpk is a bit scared of the apparent translation fragility
[01:04] <hpk> i mean it's even more fragile than the ancient py lib import hacks :-)
[01:07] <hpk> or do you think that the fragility will become less by time?
[01:07] <arigo> no, it's the RPython definition, sometimes it's hard to realize that the code we write is not RPython for some reason
[01:08] <hpk> and is it feasible to try harder to bail out with somewhat clear error message as early as possible?
[01:08] <arigo> or just forget it when we write "temporary" hack
[01:08] <arigo> like your warnings
[01:08] <arigo> :-)
[01:09] <hpk> i actually marked that as NOT_RPYTHON IIRC
[01:09] <hpk> :-)
[01:09] <arigo> clearer error messages are definitely needed
[01:10] <arigo> it's not enough to mark something as NOT_RPYTHON if it's still called at run-time...
[01:10] <hpk> but this is detected, no?
[01:11] <arigo> well, it crashes, if I'm not mistaken
[01:11] <arigo> so yes, it's detected, in a sense
[01:12] <arigo> at the moment the "correct" solution involves strange stuff
[01:12] <arigo> e.g. writing an entry in ann_override.py
[01:13] <arigo> all this is obscure and tends to change, it's not clear how stable the current way of doing things is
[01:13] <arigo> and giving reasonable error messages is definitely needed, but again at the moment it's not entierely clear what is stable and what is not
[01:14] <hpk> i understood that much :-)
[01:14] <hpk> considering that you have a pretty good capability of dealing with dynamisms, you saying this, well, umm, says something :-)
[01:16] <arigo> anyway all this mess comes from the fact we're trying to do *static* annotation :-)
[01:17] <arigo> I'm not saying dynamic annotation is necessarily easier, but it's messy in different ways
[01:19] <hpk> define dynamic annotation
[01:20] <arigo> Psyco :-)
[01:21] <hpk> i am not letting you out that easily :-)
[01:23] <hpk> aren't you maybe saying that we are not (dynamically?) specializing?
[01:23] <arigo> PyPy's annotator is completely static at the moment
[01:24] <hpk> in the sense that is has no help from runtime
[01:24] <arigo> yes, and even more, it takes an input program in-memory and does its magic, until it's complete
[01:24] <arigo> so it's not even a just-in-time annotator, it's really like running gcc
[01:25] <arigo> the purpose is to evolve it into something working on dynamic data, but it's not at all what it's now
[01:26] <arigo> e.g. right now we're working on a problem with the annotator calling W_TypeObject._freeze_(), which adds new methods to classes, which breaks the promize that classes are immutable
[01:26] <arigo> the annotator cannot call _freeze_() before it sees the particular W_TypeObject instance, but cannot see the instance before it sees the involves classes
[01:27] <pedronis> hpk: we could have written a dynamic compiler in Python for Python and run it on itself and frozen the resulting state from memory to disk
[01:27] <pedronis> it has different trade-offs
[01:27] <pedronis> for example it makes less sense for some possible target platforms
[01:30] <hpk> (side note: i think we may want to reach out to GCC and LLVV communities for the kind of expertise we look for)
[01:31] <hpk> a question:
[01:32] Action: hpk thinks some more
[01:36] <hpk> wouldn't it make sense to go for resolving the ever-persisting caching issues in a more systematical manner?
[01:37] <arigo> yes, that's precisely what I tried today
[01:37] <hpk> for example, some kind of global mechanism where each cache has to register with a "this is how to fill it" function?
[01:37] <arigo> it's introducing more problems, like the one I described above
[01:37] <arigo> hpk: that's exactly what I did indeed :-)
[01:38] <arigo> hpk: ah, almost
[01:38] <hpk> no, i mean freezing before you start to annotate
[01:38] <hpk> um, even before you generate the first flowgraph
[01:38] <arigo> there is no sane way to (1) find all cache instances (2) know all possible keys
[01:38] <arigo> or maybe there is, but it's less elegant :-)
[01:38] <hpk> i know
[01:38] <hpk> but it's a very clear and stable solution
[01:38] <arigo> no, I fear it would be extremely messy all around
[01:39] <arigo> e.g. how do you find all keys to AppLevelInterpClass's cache
[01:40] <arigo> for space.app_exec it is downright impossible
[01:40] <hpk> ok, next try: could we restart completely from the beginning after freezing caches while we go?
[01:40] <arigo> so we'd have to remove it
[01:41] <arigo> yes, but there are lots of caches :-) that would take ages, restarting whenever any one is updated
[01:41] <hpk> appexec() was not meant for runtime, i though
[01:41] <arigo> no, it's fine to use it at run-time
[01:41] <arigo> if it wasn't meant for run-time it wouldn't need to use Caches at all
[01:42] <hpk> well, at least when we came up with the appexec() idea it wasn't meant this way, but nevermind
[01:42] <hpk> i was hoping that restarting the process is not that bad if it happens very early
[01:43] <arigo> it doesn't
[01:43] <arigo> well, also note that annotation is basically in good shape, it's just a change that broke things
[01:52] <hpk> so how you are you trying to fix the W_TypeObject._freeze_ issue?
[01:54] <hpk> do you e.g. try to talk back to the annotator about e.g. which classes got changed?
[02:04] <arigo> no, we just teached the annotator when it's suspicious that a method is seen missing
[02:04] <arigo> and look again in this case
[02:05] <arigo> it's a bit strange, but seems to do the trick at the moment
[02:07] <hpk> doesn't that rely too much on the assumption that it is only the lookup of a method that could be affected by _freeze_() execution?
[02:07] <arigo> we thought about some way to talk back, or just raising "ThingsChangedRestartEverything"...
[02:08] <arigo> yes, it's very much special case...
[02:09] <hpk> obviously, ThingsChangedRestartEverything is a brutal but also pretty robust approach ...
[02:09] <hpk> ... and we would naturally try to make them happen as early as possible
[02:10] fredrik (fredrik@c83-248-135-181.bredband.comhem.se) left irc: "http://fredrikj.net";
[02:15] <arigo> possibly... for the moment the problem appears to be solved, and the problem of RestartEverything is that if each W_TypeObject raises one when it's first seen annotation will restart a lot of times
[02:28] <arigo> hpk: well, you're right, I tried to see when _freeze_s are called
[02:28] <arigo> they tend to be called near the beginning
[02:28] <arigo> it might have made sense to do try a scheme like, not throwing RestartEverything, but starting a 15-seconds alarm and restarting only after that....
[02:29] <hpk> set the just_try_to_restart_everything_next_time_when_you_get_into_trouble_flag
[02:30] <arigo> :-)
[02:30] <arigo> the real problem here was to *detect* when we get into trouble, precisely
[02:31] <arigo> i.e. detect when we're using an attribute when more about it might have shown up
[02:31] <hpk> you are in undetectable trouble if you can't determine if you are in trouble, i guess
[02:32] <hpk> i am not sure you need to detect trouble at this fine grained level
[02:33] <hpk> can't we use the fragility to assume it will just break cleanly e.g. after we did _freeze_s?
[02:35] <arigo> no, that's always the problem with the annotator
[02:35] <arigo> fragile means that it will crash in strange way
[02:35] <arigo> in this case, we found the problem because it gave more warnings than usual
[02:36] <arigo> but warnings of a kind that are not all errors, just really gcc-like warnings
[02:36] <arigo> and there are cases where the problem woulnd't even show up as this warning
[02:36] <arigo> i.e. it's fragile in the worst sense of getting wrong or unexpected results
----- silence for 21 minutes ----- [02:57] arigo (~arigo@c-3a8b70d5.022-54-67626719.cust.bredbandsbolaget.se) left irc: "good night"
[02:57] <pedronis> night
[02:58] pedronis (~Samuele_P@c-3a8b70d5.022-54-67626719.cust.bredbandsbolaget.se) left irc: "Chatzilla 0.9.68a [Firefox 1.0.2/20050317]"
[03:10] yuuh (oqeesx@82.140.29.187) left irc: "utz utz utz"
----- silence for 1 hr and 32 minutes ----- [04:42] sanxiyn (tinuviel@222.233.187.66) joined #pypy.
----- silence for 23 minutes ----- [05:05] <sanxiyn> Hello
[05:05] <sanxiyn> How do I run tests under lib-python? I checked out testresult directory, and now lost. :(
----- silence for 1 hr and 28 minutes ----- [06:33] sanxiyn (tinuviel@222.233.187.66) left irc: "Bye"
----- silence for 56 minutes ----- [07:29] sanxiyn (tinuviel@222.233.187.66) joined #pypy.
----- silence for 29 minutes ----- [07:58] idnar (mithrandi@idnar.user) left irc: Nick collision from services.
[07:58] idnar_ (mithrandi@idnar.user) joined #pypy.
----- silence for 55 minutes ----- [08:53] idnar_ (mithrandi@idnar.user) left irc: Client Quit
[08:54] idnar (mithy@idnar.user) joined #pypy.
----- silence for 21 minutes ----- [09:15] Nick change: idnar -> idnar_
----- silence for 1 hr and 43 minutes ----- [10:58] sanxiyn (tinuviel@222.233.187.66) left irc: "Bye"
----- silence for 55 minutes ----- [11:53] arigo (~arigo@c-3a8b70d5.022-54-67626719.cust.bredbandsbolaget.se) joined #pypy.
----- silence for 53 minutes ----- [12:46] <hpk> hello
[12:47] <arigo> morning
[12:56] yuuh (nnetjcjg@82.140.29.187) joined #pypy.
[12:56] <arigo> I'm trying to create a Compiler abstraction
[12:57] <arigo> (this started as an attempt to fix setup_warning for the annotator)
[12:57] Action: hpk sidenotes that he liked the discussion style yesterday night, as it wasn't so chatty but more thoughful
[12:58] <arigo> yes
[12:59] <arigo> interactive.py and in particular the stdlib codeop.py is quite messy
[12:59] sanxiyn (tinuviel@222.233.187.66) joined #pypy.
[12:59] <arigo> it's messy to know when an interactively typed code snippet is complete
[12:59] <arigo> hi seo!
[12:59] <hpk> sanxiyn: hi seo!
[13:00] <sanxiyn> hi
[13:00] <hpk> sanxiyn: you run tests under lib-python by starting 'py.test' or ../pypy/test_all.py in lib-python (or for best results directly in lib-python/2.3.4/test/)
[13:00] <hpk> it doesn't work at all on windows though
[13:01] <sanxiyn> I'm on Linux now.
[13:02] <sanxiyn> Eh, how long does it take? :-)
[13:02] <hpk> you will then create files under pypy-dist/testresult/youruser@yourhost and you need to 'svn add' your specific directory
[13:03] <hpk> you have options: run e.g. with 'py.test -k core' to run all core tests or 'py.test -k "core err"' to run all core tests that currently give errors
[13:04] <sanxiyn> Okay.
[13:04] <hpk> sorry, that needs to be 'py.test -k "core error"'
[13:05] <hpk> you can look at http://codespeak.net/~hpk/pypy-testresult/ to see how long tests run
[13:06] <hpk> the keyword selection actually looks at your 'testresult' directory contents so it makes sense to 'svn up' it regularly
[13:07] <arigo> hpk: do you mind if I forward my mails to arigo@codespeak.net and store them there?
[13:07] <hpk> arigo: no
[13:08] <sanxiyn> hpk: How do I run a single test?
[13:08] <sanxiyn> Eh, must be py.test <file>. Stupid me.
[13:09] <hpk> yes, it's a bit (too?) clever, though, because even if you 'py.test 2.3.4/test/test_class.py' it will actually use 'modified-2.3.4/test/test_class.py' but you see that easily on the output that ensues
[13:11] <hpk> arigo: do you need procmail?
[13:11] <sanxiyn> Eh.
[13:11] <arigo> hpk: I guess so
[13:11] <sanxiyn> pypy-dist/lib-python$ py.test 2.3.4/test/test_xmlrpc.py
[13:11] <sanxiyn> E assert resultdir.check(dir=1)
[13:11] <sanxiyn> > assert local('/home/tinuviel/src/pypy/dist/testresult/tinuviel@debian').check(dir=1)
[13:12] <sanxiyn> Should I create a directory?
[13:12] <hpk> no, usually not, it should create it for you
[13:12] <hpk> which function?
[13:12] <sanxiyn> It didn't.
[13:12] <hpk> arigo: done, let me know if you are missing anything
[13:13] <arigo> thanks
[13:13] <arigo> I'm confused, where do the mails arrive?
[13:13] <hpk> IIRC .maildir/
[13:13] <hpk> it's maildir format, obviously
[13:13] <hpk> you can use procmail to put it into a mailbox if you prefer that
[13:13] <arigo> ah
[13:13] <arigo> ok thanks
[13:15] <arigo> I don't see any test mail arriving, though
[13:15] <hpk> sanxiyn: can you 'svn up' and retry? i think i fixed it
[13:17] <hpk> arigo: it's forwarded somehow to arigo@tunes.org, let me check
[13:17] <arigo> the test mails don't show up on arigo@tunes.org either, hence my confusion
[13:18] <sanxiyn> hpk: Yes, your fix helped.
[13:18] <sanxiyn> Thanks.
[13:18] <hpk> they were delivered to arigo@tunes.org though, maybe landed in some strange folder?
[13:18] <arigo> no, I'm checking the global backup copy folder on the tunes.org machine directly
[13:18] <hpk> arigo: try again
[13:19] <arigo> ah
[13:19] <arigo> ok
[13:19] <arigo> thanks
[13:21] <hpk> arigo: can you make a .forward entry with '\arigo'?
[13:22] <arigo> ? ok
[13:22] <hpk> i am probably going to setup .forward's for everyone we know mail addresses of so that mail to user@codespeak.net arrives
[13:22] <hpk> this is nice because people may respond to commit mailing lists
[13:23] <hpk> and not all of them have a Reply-To set
[13:23] <hpk> and i don't want mail to pile up in user's directories when they never read it
[13:23] <arigo> where is the .forward file documented?
[13:24] <hpk> "man local" on codespeak, it's a postfix feature (or sendmail for that matter) IIRC
[13:25] <sanxiyn> Yes, it is an MTA feature.
[13:26] <arigo> I'm veery confused, the previous mails arrived to tunes.org but the mails there weren't copied to the backup folder for the last 6 days any more
[13:27] <arigo> this backup folder is very close to 50MB, I wonder if there is some stupid limit
[13:27] <sanxiyn> Running test_xmlrpc with --oldstyle results in failures in different places, and increasing elapsed time from 30s to 300s...
[13:27] <arigo> if so I've probably lost my e-mails between yesterday evening and now
[13:27] <sanxiyn> (btw, it should be run with --oldstyle, because it dispatches on types.InstanceType.)
[13:29] <hpk> arigo: :-(
[13:29] <hpk> sanxiyn: mark it as oldstyle then in conftest.py
[13:29] yuuh (nnetjcjg@82.140.29.187) left irc: Read error: 110 (Connection timed out)
[13:29] <sanxiyn> hpk: Yup.
[13:30] <sanxiyn> hpk: By the way, now I can see test_xmlrpc.txt in my testresult directory, but it says "options: []" even if it was produced with --oldstyle. Is this normal?
[13:31] <hpk> it's slightly inconsistent, unfortuantely, but it actually run with oldstyle ... wait a moment
[13:33] <arigo> hpk: what does the \arigo mean in .forward?
[13:33] <hpk> arigo: i think it means 'deliver locally, don't go through more alias resolving'
[13:34] <arigo> ah
[13:34] <hpk> i have used it for years, "man local()" you can just leave it empty, but i am not sure that is true for sendmail as well
[13:47] <hpk> sanxiyn: 'svn up', it should be fixed now
[13:47] <hpk> thanks for pointing the problem out
[13:52] <arigo> hpk: confused again
[13:52] <arigo> the .forward seems to be partly ignored
[13:53] <hpk> arigo: let's go to #arigo
[13:56] <sanxiyn> hpk: Your fix seems to work. Thanks!
----- silence for 4 hr and 26 minutes ----- [18:22] idnar_ (mithy@idnar.user) left irc: "kbye"
----- silence for 45 minutes ----- [19:07] aleale (~root@cpe.atm0-0-0-129140.0x3ef2fa3a.bynxx3.customer.tele.dk) joined #pypy.
----- silence for 23 minutes ----- [19:30] idnar (mithrandi@idnar.user) joined #pypy.
----- silence for 1 hr and 17 minutes ----- [20:47] tewk (tewk@192.41.88.101) left #pypy.
----- silence for 1 hr and 14 minutes ----- [22:01] yuuh (hrpmgbf@i528C1E27.versanet.de) joined #pypy.
----- silence for 33 minutes ----- [22:34] aleale (~root@cpe.atm0-0-0-129140.0x3ef2fa3a.bynxx3.customer.tele.dk) left irc: "*monkey grin*"
[22:38] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) left irc: "Leaving"
[22:39] dialtone (~dialtone@host111-56.pool80117.interbusiness.it) joined #pypy.
----- silence for 1 hr and 16 minutes ----- [23:55] sanxiyn (tinuviel@222.233.187.66) left irc: "Bye"
[00:00] --- Fri May 6 2005