| cfbolz (n=cfbolz@p54AB8AF2.dip0.t-ipconnect.de) left irc: "Leaving" | 00:20 | |
| pedronis (n=Samuele_@c-3c8b70d5.022-54-67626719.cust.bredbandsbolaget.se) left irc: "Chatzilla 0.9.68a [Firefox 1.0.4/20050511]" | 00:22 | |
| rhymes (n=rhymes@host126-5.pool80183.interbusiness.it) left irc: "supercalifragiliblahblah" | 01:26 | |
| paniq_ (n=paniq@port-212-202-49-191.dynamic.qsc.de) joined #pypy. | 01:44 | |
| Rhamphoryncus (n=rhamph@unaffiliated/rhamphoryncus) joined #pypy. | 03:11 | |
| ita (n=ita@kde/developer/tnagy) joined #pypy. | 03:19 | |
| ita | compiling 0.8 atm here | 03:19 |
|---|---|---|
| ita | it is huge | 03:30 |
| ita | wow | 03:30 |
| ita | In file included from testing_1.c:1: | 03:53 |
| ita | common_header.h:4:19: error: gc/gc.h: | 03:53 |
| ita | no such file or directory mmm | 03:54 |
| ita (n=ita@kde/developer/tnagy) left irc: "Konversation terminated!" | 04:01 | |
| _hannes (n=yuuhu@i577B40E0.versanet.de) left irc: Read error: 104 (Connection reset by peer) | 05:16 | |
| stakkars (n=tismer@adsl-71-133-157-81.dsl.irvnca.pacbell.net) joined #pypy. | 07:21 | |
| xyz359 (n=xyz359@pool-68-238-174-197.tampfl.fios.verizon.net) left irc: ";)" | 07:31 | |
| idnar (i=mithrand@unaffiliated/idnar) left irc: Nick collision from services. | 07:45 | |
| idnar_ (i=mithrand@unaffiliated/idnar) joined #pypy. | 07:45 | |
| guido_wesdorp (n=johnnyde@a80-127-80-154.adsl.xs4all.nl) joined #pypy. | 07:53 | |
| guido_w (n=johnnyde@a80-127-80-154.adsl.xs4all.nl) left irc: Read error: 110 (Connection timed out) | 08:10 | |
| arigo (n=arigo@c-3c8b70d5.022-54-67626719.cust.bredbandsbolaget.se) joined #pypy. | 10:13 | |
| rhymes (n=rhymes@host126-5.pool80183.interbusiness.it) joined #pypy. | 10:42 | |
| cfbolz (n=cfbolz@p54AB9DD2.dip0.t-ipconnect.de) joined #pypy. | 11:29 | |
| cfbolz | hi! | 11:29 |
| jacob22|home | tic: ANTLR may be what you want. | 13:11 |
| tic | jacob22|home, thanks! I'll look it up | 13:29 |
| xorAxAx | does anyone know if cpython mmaps pycs? | 13:30 |
| xorAxAx | java does, and so should python :) | 13:30 |
| cfbolz | I am pretty sure that it doesn't | 13:30 |
| xorAxAx | the core team says that you should go multiprocess if the GIL is your bottleneck. and they havent even implemented this ... | 13:31 |
| Rhamphoryncus | xorAxAx: why would it? To avoid copying them? | 13:31 |
| xorAxAx | Rhamphoryncus: in order to optimise mem usage for multiple py processes | 13:32 |
| Rhamphoryncus | ahhh good point | 13:32 |
| xorAxAx | for a big thing, this should decrease the mem usage significantly | 13:32 |
| Rhamphoryncus | maybe | 13:33 |
| cfbolz | big thing meaning thousands of processes? | 13:33 |
| xorAxAx | no, meaning a few processes but a big source tree | 13:33 |
| cfbolz | ah | 13:34 |
| Rhamphoryncus | it would help slightly even for single processes. Effectively a smaller memory footprint since it would use only the kernel's cache of the file, and not a copy on the heap | 13:34 |
| xorAxAx | yeah, which has better paging behaviour | 13:35 |
| Rhamphoryncus | yup | 13:36 |
| cfbolz | feel free to submit a patch, I would say :-) | 13:36 |
| Rhamphoryncus | hehe | 13:36 |
| xorAxAx | i just realise that you would need to write c | 13:36 |
| xorAxAx | because you need a lazy read | 13:36 |
| cfbolz | oh yes | 13:36 |
| Rhamphoryncus | why does that need C? | 13:36 |
| xorAxAx | because the inner interpreter would need to read code chunks from a file instead of jumping in heap | 13:38 |
| xorAxAx | but considering that - it would not help because those marshal blobs are deallocated after loading i guess | 13:38 |
| xorAxAx | so this idea is completly useless | 13:38 |
| Rhamphoryncus | heh | 13:38 |
| xorAxAx | the func objects are stored anyway on the heap, regardlessy where they were read from | 13:39 |
| Rhamphoryncus | yeah | 13:40 |
| Rhamphoryncus | I don't know how much of it is stored as a string anyway | 13:40 |
| cfbolz | arigo: what are the two objectspaces supposed to do? | 13:52 |
| arigo | cfbolz: force the 'space' to be kept around everywhere, to measure the performance impact | 13:57 |
| cfbolz | yes sure | 13:58 |
| arigo | nothing more :-) | 13:58 |
| cfbolz | but they don't actually provide any difference | 13:58 |
| cfbolz | ok | 13:58 |
| alextp (n=top@201008227201.user.veloxzone.com.br) joined #pypy. | 14:00 | |
| arigo | it doesn't compile because of a limitation of specialization, which is mostly not working with methods | 14:01 |
| arigo | space.wrap -> crash | 14:01 |
| cfbolz | yes, I read the checkin message. is it hard/worth fixing it? | 14:02 |
| arigo | worth fixing it, yes | 14:02 |
| arigo | hard, yes too | 14:02 |
| arigo | :-) | 14:02 |
| cfbolz | :-) | 14:02 |
| arigo | ah, it seems that having only one real space and 'space=None' as alternative doesn't trigger this specialization problem | 14:02 |
| arigo | but it should force 'space' to exist everywhere in the C code | 14:02 |
| cfbolz | hahaha | 14:02 |
| cfbolz | a bit cheating, though | 14:03 |
| arigo | of course | 14:03 |
| cfbolz | but valid for the performance benchmark | 14:04 |
| arigo | yes. | 14:05 |
| xyz359 (n=xyz359@pool-68-238-174-197.tampfl.fios.verizon.net) joined #pypy. | 14:29 | |
| arigo | waaaa, doesn't work | 14:32 |
| cfbolz | why not? | 14:33 |
| cfbolz | crashes terribly? or does not compile? | 14:33 |
| arigo | no no it compiles | 14:33 |
| arigo | but the code still has no 'space' anywhere | 14:33 |
| arigo | after 'space.call_function(w_entry_point)', the annotator knows that 'space' cannot be None any more | 14:33 |
| cfbolz | magic powers? | 14:33 |
| arigo | yes | 14:34 |
| cfbolz | damn | 14:34 |
| Action: arigo tries a more obscure hack | 14:38 | |
| arigo | for now, the RTyper will believe that SomePBC({space}) needs a MultipleFrozenPBCRepr instead of a SinglePBCRepr... | 14:39 |
| cfbolz | :-) | 14:39 |
| cfbolz | how about making the annotator dumber, just for this exercise: like removing the knowlegde that a getattr on something-or-None asserts that it cannot really be None? | 14:39 |
| arigo | it's more indirect | 14:40 |
| cfbolz | ? | 14:40 |
| arigo | because it's a method call, the annotation of 'self' is entierely rebuilt as a SomeInstance() with the precise class where the method was found | 14:40 |
| cfbolz | uh, I see | 14:40 |
| arigo | yes, it's the call, not the getattr | 14:40 |
| cfbolz | I see | 14:41 |
| cfbolz | is my impression wrong that the release didn't attract much attention? | 14:41 |
| xorAxAx | at least i did a lot of advertising :) | 14:42 |
| cfbolz | not more than usual | 14:42 |
| xorAxAx | why should it attract more attention? | 14:54 |
| cfbolz | no, I am saying it attracted less | 14:55 |
| alextp (n=top@201008227201.user.veloxzone.com.br) left #pypy. | 14:55 | |
| alextp (n=top@201008227201.user.veloxzone.com.br) joined #pypy. | 14:55 | |
| xorAxAx | the only visible difference is a speed increase for the average guy, right? | 14:56 |
| cfbolz | translatable thunks | 14:56 |
| xorAxAx | "and it is still damn slow" or something like that they could say | 14:57 |
| xorAxAx | yeah, but who needs the thunkspace? :) | 14:57 |
| cfbolz | yes, but they also could have said that for earlier releases | 14:57 |
| xorAxAx | they did :) | 14:57 |
| cfbolz | yes, but is beside the point | 14:57 |
| xorAxAx | my point is that most guys are bored if they see the "feature list". "we have gotten faster, but still 10 times slower than ordinary python". i think they are not interested in the other parts of the pypy project - at least those who didnt know pypy before | 15:02 |
| cfbolz | you have no idea. just read the comp.lang.python threads after the earlier releases. some of it was (as usual) crazy, but speed was not the most important discussion point | 15:03 |
| idnar_ (i=mithrand@unaffiliated/idnar) left irc: Read error: 110 (Connection timed out) | 15:03 | |
| xorAxAx | cfbolz: which one was, then? | 15:04 |
| cfbolz | flexibility | 15:04 |
| cfbolz | (what else) | 15:04 |
| xorAxAx | yeah, sure | 15:05 |
| xorAxAx | is there any new flexibility additions besides some new partly finished backends and the thunkspace compilation possiblity? | 15:05 |
| xorAxAx | (that need to be discussed) | 15:05 |
| jacob22|home | cfbolz: I think it is a bit too early to tell about what attention the release attracts. This week and weekend is fairly holiday intensive. Halloween in the US and All Sints Day in many parts of Europe. | 15:06 |
| cfbolz | jacob22|home: maybe | 15:06 |
| cfbolz | xorAxAx: are you kidding? | 15:06 |
| cfbolz | the parser/compiler is _very_ important | 15:06 |
| xorAxAx | cfbolz: ah, right | 15:06 |
| jacob22|home | xorAxAx: Indeed, it is the whole reason for making the release. | 15:07 |
| xorAxAx | ok | 15:07 |
| jacob22|home | So, now people can go write their own language modification in Python and they can write up their trial implementations for PEPs in Python. | 15:09 |
| xorAxAx | in fact, i cannot find any reply at all re the release announcements via gmane or google | 15:12 |
| cfbolz | right | 15:12 |
| xorAxAx | but i dont see that usually for moinmoin either :) | 15:13 |
| cfbolz | :-) | 15:14 |
| Nick change: paniq_ -> paniq | 15:15 | |
| Rhamphoryncus | hrm. Alas, my favorite threading design (using sandboxes) is simply not possible with CPython, so there's no point doing it with PyPy unless PyPy is replacing CPython | 16:20 |
| xorAxAx | what are sandboxes? | 16:23 |
| xorAxAx | in this context | 16:23 |
| Rhamphoryncus | basically seperate interpreters that can't access the outside world except by passing immutable types (such as str and int) and thread-safe types (such as queues) | 16:24 |
| Rhamphoryncus | If you use a tracing GC (ie Boehm) then you don't need the GIL | 16:25 |
| Rhamphoryncus | and there's no undefined behavior (since simultaneous access to mutable types is prohibited) | 16:26 |
| Rhamphoryncus | Contrast that to standard Python threads where the language specs don't specify interleaving or the memory model (actually, do they specify threads at all?) The CPython implementation does treat most C functions as effectively atomic, but that's pretty hacky | 16:30 |
| arigo | this is indeed a sane approach, but it requires some careful language design | 16:46 |
| Rhamphoryncus | somewhat, yeah :) | 16:49 |
| Rhamphoryncus | Especially if you try to extend it to make it possible for user classes to become immutable | 16:49 |
| arigo | indeed | 16:50 |
| arigo | probably modules too -- messy | 16:50 |
| arigo | I would say that you can already do something along these lines with py.execnet | 16:50 |
| arigo | well, not the sharing part, but the exchanging-only-primitive-objects part | 16:51 |
| Rhamphoryncus | yeah probably | 16:54 |
| arigo | PyPy could provide an in-process equivalent to execnet, I suppose, where you can really share the large blobs of immutable data between threads | 16:55 |
| xorAxAx | and there is this erlang approach | 16:55 |
| Rhamphoryncus | erlang was part of my inspiration :) | 16:55 |
| xorAxAx | arigo: but how would this fit into the language spec? | 16:55 |
| arigo | xorAxAx: I don't know :-) | 16:56 |
| xorAxAx | arigo: ;-) | 16:56 |
| arigo | at worst, you could provide a completely unsafe thread module | 16:56 |
| arigo | and let Python libraries build safe models on top of it | 16:56 |
| Rhamphoryncus | yum C ;) | 16:57 |
| xorAxAx | hmm, why is the GIL needed at all ... | 16:57 |
| xorAxAx | (for pypy) | 16:57 |
| arigo | ? for the same reason as in CPython | 16:57 |
| xorAxAx | cpython has thread-unsafe code on the c side | 16:58 |
| Rhamphoryncus | what you need is a memory model, that specifies how things interact. The GIL provides a very simple memory model | 16:58 |
| xorAxAx | does pypy has thread-unsafe code as well? | 16:58 |
| xorAxAx | s/has/have/ | 16:58 |
| xorAxAx | i mean internally | 16:58 |
| Rhamphoryncus | however, the GIL in CPython only makes C code behave sanely. It does not protect Python code | 16:58 |
| xorAxAx | an unsafe thread module would not work for cpython, i guess | 16:59 |
| xorAxAx | would it? | 16:59 |
| Rhamphoryncus | IMO the *current* thread module doesn't work ;) | 16:59 |
| Rhamphoryncus | it prevents undefined corruption, but still permits defined corruption, if that makes sense :) | 17:00 |
| xorAxAx | arigo: are the answers yes and no? | 17:01 |
| arigo | I'm not sure I understand what you mean by having thread-unsafe code on the C side | 17:02 |
| xorAxAx | code is unsafe if it relies on globals so to say | 17:03 |
| _hannes (n=yuuhu@i577B52AD.versanet.de) joined #pypy. | 17:03 | |
| xorAxAx | (modifying them potentially) | 17:03 |
| arigo | it's not only about globals | 17:03 |
| xorAxAx | assuming that there is no ref counting | 17:03 |
| Rhamphoryncus | anything is thread-unsafe if it can be affected multiple threads. Funny that :) | 17:03 |
| xorAxAx | what would happen if python didnt have a GIL and the threads used locks to syncronise python globals? | 17:04 |
| arigo | then you would crash on synchronuous access to any mutable object | 17:04 |
| arigo | in the heap | 17:05 |
| xorAxAx | on read access? | 17:05 |
| arigo | when one thread reads and another writes, or two threads write | 17:05 |
| Rhamphoryncus | right, you need to lock every mutable object as well. That includes the weakref list btw | 17:05 |
| arigo | yes, nothing new so far | 17:06 |
| xorAxAx | arigo: because the old objects were deallocated when the thread 2 put his new element into the list (example), right? | 17:06 |
| arigo | no, just any concurrent writes can leave the object in an inconsistent state | 17:06 |
| xorAxAx | hmm, yeah | 17:07 |
| arigo | or a read in the middle of a write by another thread sees an inconsistent object | 17:07 |
| xorAxAx | how about an object field locked then? | 17:07 |
| Rhamphoryncus | And you'd get a "crash" if you needed a couple operations to be treated as a single (atomic) operation and they're not. You wouldn't get a segfault, but you would (probably, eventually!) get an exception | 17:07 |
| arigo | no, you can very easily get segfaults | 17:07 |
| xorAxAx | would it be difficult to implement it in pypy? such an object field? | 17:07 |
| Rhamphoryncus | err.. right, we're talking about unsafe C here. Forgot :) | 17:08 |
| Rhamphoryncus | xorAxAx: it took java *years* to get it right, going through many revisions | 17:08 |
| xorAxAx | Rhamphoryncus: you mean the syncronisation primitives visible to the programmer? | 17:08 |
| Rhamphoryncus | PyPy could copy Java, making it much quicker, but it'd still be difficult | 17:08 |
| Rhamphoryncus | no, to come up with a memory model that let you do anything sane | 17:09 |
| xorAxAx | Rhamphoryncus: and what was the clue they needed to accomplish that? | 17:09 |
| Rhamphoryncus | really the WHOLE threading design | 17:09 |
| Rhamphoryncus | eh? | 17:09 |
| xorAxAx | i mean, what is the key point of their model? | 17:09 |
| Rhamphoryncus | I don't think there is one key point | 17:10 |
| Rhamphoryncus | it's big and complicated | 17:10 |
| Rhamphoryncus | oh, and it reduces optimization potential ;) | 17:10 |
| xorAxAx | is there a page describing it well enough? | 17:12 |
| Rhamphoryncus | lots I believe :) | 17:13 |
| Rhamphoryncus | google for the "double checked lock problem" | 17:13 |
| Rhamphoryncus | used for years, even in books on java.. turns out it doesn't work ;) | 17:14 |
| Action: xorAxAx found http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html already | 17:14 | |
| xorAxAx | hmm, thats a result of jitting | 17:16 |
| Rhamphoryncus | the Boehm GC is pretty much the same btw. C does *not* support it, but it "mostly works", so people use it.. | 17:16 |
| Rhamphoryncus | (and that's why I can't do sandboxes in CPython. Needs Boehm) | 17:17 |
| xorAxAx | hmm, the only solution is a TLS pattern | 17:17 |
| xorAxAx | and the volatile modifier | 17:17 |
| xorAxAx | but this is another level than the issue pypy has .. | 17:18 |
| xorAxAx | pypy hasn't even syncronised "transactions on objects" internally (like cpython) | 17:18 |
| Rhamphoryncus | volatile isn't sufficient. It doesn't guarantee the pointer will be kept around | 17:19 |
| xorAxAx | "does not work" - oops | 17:19 |
| xorAxAx | so just TLS works | 17:19 |
| Rhamphoryncus | TLS? | 17:19 |
| xorAxAx | thread local storage | 17:19 |
| Rhamphoryncus | hrm. Not sure what you mean though :) | 17:20 |
| xorAxAx | you store a flag which is just accessible from the ucrrent thread | 17:20 |
| xorAxAx | tls means that some variable can have different values in different threads | 17:21 |
| xorAxAx | s/some/a/ | 17:21 |
| xorAxAx | there is an example on the page above | 17:21 |
| Rhamphoryncus | yes, but.. how does that fix threading? :) I know how TLS works.. | 17:21 |
| Rhamphoryncus | btw, I think Boehm's support of TLS is iffy | 17:22 |
| xorAxAx | see the pattern | 17:22 |
| Action: Rhamphoryncus looks at the page | 17:23 | |
| Rhamphoryncus | ahh, as a work-around for double checked locking. I should pay more attention ;) | 17:23 |
| xorAxAx | yeah | 17:24 |
| xorAxAx | but as i said - this doesnt help pypy :) | 17:24 |
| Rhamphoryncus | hehe | 17:24 |
| Rhamphoryncus | After reading about those problems do you agree with the approach of sandboxes, where you have no undefined behavior? | 17:25 |
| xorAxAx | arigo: how about implementing this boolean flag in every object and implementing some CMPXCHG logic (atomic lock checking) into the backends? | 17:26 |
| xorAxAx | Rhamphoryncus: yeah, they just dont work for python :) | 17:26 |
| Rhamphoryncus | *CPython*. They can work if you don't use C ;) | 17:26 |
| xorAxAx | Rhamphoryncus: i mean, how would such a sandbox start? | 17:26 |
| xorAxAx | no, i mean the language | 17:26 |
| xorAxAx | Rhamphoryncus: empty? | 17:27 |
| Rhamphoryncus | I've been debating how best to do that | 17:27 |
| Rhamphoryncus | Most likely it'd start with a function call | 17:27 |
| Rhamphoryncus | The function itself, the globals, and the arguments would have to be immutable | 17:28 |
| Rhamphoryncus | You could also have monitor sandboxes, where the sandbox persists but only one call can be within it at a time | 17:28 |
| xorAxAx | and how do you make them immutable? | 17:29 |
| Rhamphoryncus | and threaded sandboxes.. used to start threads when you don't have a GIL-based threading module to fall back on ;) | 17:29 |
| Rhamphoryncus | str and int are immutable to begin with, functions can be too.. the tricky part is the module that contains the function | 17:30 |
| arigo | xorAxAx: removing the GIL in Python is not related to this Java-level discussion... | 17:30 |
| Rhamphoryncus | *probably* use a freeze() function to do that | 17:30 |
| arigo | xorAxAx: see for example the very long discussions on python-dev, | 17:30 |
| arigo | xorAxAx: like http://mail.python.org/pipermail/python-dev/2005-September/056423.html | 17:30 |
| xorAxAx | yeah, i know! | 17:30 |
| Rhamphoryncus | arigo: those get funny after you've read enough of them :) | 17:31 |
| cfbolz | :-) | 17:31 |
| Rhamphoryncus | There's even a post of mine on sandboxes | 17:31 |
| xorAxAx | arigo: MvL says that the idea is nice | 17:31 |
| xorAxAx | arigo: just that its not cpython compatible | 17:31 |
| xorAxAx | because it would be incompatible | 17:31 |
| xorAxAx | to old modules etc. and a lot of owrk | 17:32 |
| Rhamphoryncus | heh, my sandboxes are compatible, sort of. They let you use existing modules in normal code, you just can't pass the modules in to the sandboxes :) | 17:32 |
| xorAxAx | but pypy is flexible, i read somewhere %-) | 17:33 |
| Rhamphoryncus | I may try implementing sandboxes in PyPy eventually. Wait for the performance to get up higher first though :) | 17:34 |
| xorAxAx | why? | 17:35 |
| Rhamphoryncus | why wait? | 17:35 |
| xorAxAx | yes, why wait? | 17:35 |
| Rhamphoryncus | laziness :) | 17:35 |
| xorAxAx | in fact, pypy is very complex - thats a good reason as well :) | 17:36 |
| Rhamphoryncus | and really sandboxes don't add functionality. They impose restrictions to allow greater confidence in threading and to allow optimizations. Obviously optimizations aren't applicable if pypy is slow | 17:36 |
| cfbolz | xorAxAx: depends of course, what you mean with "pypy" in that sentence. the standard interpreter is reasonably easy | 17:36 |
| xorAxAx | Rhamphoryncus: do you know this erlang threading semantics module for python? | 17:37 |
| Rhamphoryncus | hrm I never looked too close into it | 17:37 |
| cfbolz | candygram | 17:37 |
| xorAxAx | cfbolz: hmm, would you need to modify the interpreter to implement this object level locking and not the object space? | 17:38 |
| cfbolz | the standard interpreter includes the object space | 17:38 |
| cfbolz | but yes | 17:38 |
| xorAxAx | hmm, ok | 17:38 |
| Rhamphoryncus | object-level locking requires defining wht an "object" is. Python is rather vague on that, and is likely to get even more vague ;) | 17:40 |
| xorAxAx | it is an implementation detail in any case ... | 17:41 |
| xorAxAx | and mutable object is kinda clear for python, isnt it? | 17:41 |
| Rhamphoryncus | Guido has mentioned removing the enforcement on class methods, letting you assign any function to a class and having it get bound and called for instances | 17:41 |
| Rhamphoryncus | for CPython it is | 17:41 |
| Rhamphoryncus | if defined in C.. | 17:42 |
| Rhamphoryncus | If it's a complicated object done in Python.. well it'll end up looking like RMS came up with the name ;) | 17:42 |
| xorAxAx | which name? | 17:42 |
| xorAxAx | richard "root mean square" stallman? | 17:43 |
| Rhamphoryncus | I'm thinking HURD here | 17:44 |
| Rhamphoryncus | or HIRD.. Hurd of Interfaces Representing Depth | 17:44 |
| Rhamphoryncus | consider this.. you have a class for a tree structure, and it has a .foreach() method. Does it remain locked while in the .foreach() method? | 17:45 |
| Rhamphoryncus | And if that's not bad enough consider finalizers! What if one of them tried to modify the tree structure. Is it locked at the time? Where is the finalizer getting called from? | 17:46 |
| Rhamphoryncus | I'm actually leaning towards java on that one, atleast their weakref handling.. it doesn't call anything, it just adds it to a list that the application is expected to poll occasionally | 17:47 |
| xorAxAx | Rhamphoryncus: that is not relevant for pypy | 17:47 |
| xorAxAx | locking to ensure object integrity is not related to synchronisation directives on the language layer | 17:48 |
| Rhamphoryncus | Why? Because pypy distinguishes "core" types from user-level types? | 17:48 |
| xorAxAx | no, because it should not e.g. lock the list for you but just the list header | 17:48 |
| xorAxAx | you need to lock the elements on your own | 17:49 |
| Rhamphoryncus | You're implying manual locking | 17:49 |
| Rhamphoryncus | I thought the intent was to use automatic per-object locks | 17:49 |
| xorAxAx | yeah, for internal integrity | 17:50 |
| xorAxAx | btw, what i have never understood is this: cpython switches the thread every x words. aList[:2] = ... compiles to more than word. why is it thread-safe, though? | 17:51 |
| xorAxAx | s/than word/than one word/ | 17:51 |
| Rhamphoryncus | the aList[:2] = is actually one call to a C function | 17:52 |
| xorAxAx | in fact, you have to use locks in python core more often than many python programmers think. the question is: at which point does the GIL help the python programmer at all? | 17:52 |
| Rhamphoryncus | The real nasty thing is that Py_DECREF may call __del__ methods which may do anything, so it can switch within t hem | 17:53 |
| xorAxAx | Rhamphoryncus: those are 4 vm steps | 17:53 |
| xorAxAx | 3x load and one store_slice | 17:53 |
| Rhamphoryncus | load of what? the 2? the ...? | 17:54 |
| xorAxAx | the operand on the right side (...), the list and the constant in the slice arg | 17:54 |
| Rhamphoryncus | and I'd say the GIL *doesn't* help the python program. It provides you with defined corruption, rather than the undefined corruption you'd get in C. It's still corruption though | 17:54 |
| xorAxAx | yeah, but you often read in clp that the "GIL protects you here" | 17:55 |
| Rhamphoryncus | Well, you're right that it doesn't make them all atomic. Only the last part is done atomicly | 17:55 |
| Rhamphoryncus | That's because often you only need the last part to be atomic :) | 17:55 |
| Rhamphoryncus | I think it's a piss-poor design if you rely on the GIL though ;) | 17:55 |
| xorAxAx | no, you hope that it is | 17:55 |
| jacob22|home | Any sane solution to the threading problem requires semantic changes to Python. | 17:55 |
| Rhamphoryncus | jacob22|home: sandboxed threads are my favorite option ;) | 17:56 |
| jacob22|home | Right now a thread owns nothing and has access to everything. | 17:56 |
| xorAxAx | jacob22|home: but the one with a field in the object header, suggested by the posting by arigo above could not even be defeated by MvL | 17:56 |
| xorAxAx | and it would not change the semantics | 17:57 |
| jacob22|home | xorAxAx: The field would indicate ownership? | 17:57 |
| xorAxAx | jacob22|home: to indicate the locking state | 17:58 |
| jacob22|home | xorAxAx: And how would that be checked by others trying to access the object? | 17:58 |
| xorAxAx | jacob22|home: threads wait till it has dropped to zero before modifying the object | 18:00 |
| jacob22|home | Isn't that a semantic change? | 18:00 |
| xorAxAx | in which scenario? | 18:01 |
| xorAxAx | i dont think so | 18:01 |
| xorAxAx | the coder would not see this at all | 18:01 |
| Rhamphoryncus | unless it deadlocks ;) | 18:02 |
| xorAxAx | yep | 18:02 |
| xorAxAx | but it cannot deadlock if pypy has no bugs, right? | 18:02 |
| cfbolz | ? | 18:03 |
| Rhamphoryncus | That assumes it's only usable for core types, in which case I regard it as entierly useless :) | 18:03 |
| jacob22|home | Rhamphoryncus: Actually, unless the resource isn't released. And then we need some way of making sure we can get out of such a situation. | 18:03 |
| xorAxAx | because the locked transaction is low level and would not involve calling code that needs to lock this or other objects | 18:03 |
| Rhamphoryncus | jacob22|home: there lies madness | 18:03 |
| xorAxAx | Rhamphoryncus: why just for core types? | 18:03 |
| Rhamphoryncus | CPython *could* use per-object locks. They'd be a nightmare to maintain though | 18:04 |
| Rhamphoryncus | <xorAxAx> but it cannot deadlock if pypy has no bugs, right? | 18:04 |
| jacob22|home | Rhamphoryncus: I don't think so. There are languages with primitives to deal with such problems. SR is the best one I know of. | 18:04 |
| xorAxAx | Rhamphoryncus: just the things in the std. object space need support for this ll thing i think | 18:04 |
| Rhamphoryncus | I have confidence in the user. If it COULD deadlock, it WILL deadlock ;) | 18:04 |
| Rhamphoryncus | jacob22|home: SR? | 18:05 |
| jacob22|home | Rhamphoryncus: Everything can deadlock. | 18:05 |
| Rhamphoryncus | jacob22|home: actually some languages can't. They can still live lock though | 18:05 |
| xorAxAx | jacob22|home: things can just deadlock if there a two syncronised objects that could wait on each other at all | 18:05 |
| jacob22|home | Rhamphoryncus: Synchroizing Resources. Research language out of the University of Arizona. Quite old by now. | 18:05 |
| xorAxAx | jacob22|home: but if the locked transactions (LOCs) are such small that you dont call code that fiddles with other objects ... | 18:06 |
| Rhamphoryncus | ahh | 18:06 |
| Rhamphoryncus | heh, there's sooooooo many different designs for concurrency, once you start digging | 18:06 |
| xorAxAx | so the question is: why would you need to touch more than one object in the locked state? | 18:07 |
| jacob22|home | xorAxAx: You have to in order to guarantee atomicity. | 18:07 |
| xorAxAx | jacob22|home: make an example | 18:07 |
| xorAxAx | e.g. a list.pop - it would just need to touch the list | 18:08 |
| xorAxAx | while locking it | 18:08 |
| jacob22|home | We have a banking system. You deduct $1 from one customer account and add it to another one. | 18:08 |
| xorAxAx | jacob22|home: i am not talking about transactions in the python code | 18:08 |
| xorAxAx | jacob22|home: i am talking about python objects (the interpreter) | 18:09 |
| Rhamphoryncus | I assert that any system that only protects core types is not worth doing | 18:09 |
| xorAxAx | Rhamphoryncus: depends on your goal. the one i am thinking about is: the GIL has to die | 18:09 |
| xorAxAx | not about language semantics or design | 18:10 |
| Rhamphoryncus | xorAxAx: killing the GIL is a means to an end, not an end unto itself | 18:10 |
| jacob22|home | xorAxAx: Ahh. Well, the interpreter can be a lot more primitive, since we have control over what is going on. | 18:10 |
| xorAxAx | Rhamphoryncus: sure, it will enable perfect threading and scalabilit | 18:11 |
| xorAxAx | y | 18:11 |
| xorAxAx | jacob22|home: yep | 18:11 |
| Rhamphoryncus | the GIL prevents using multiple CPUs and doesn't help user code use threads safely. Per-object locks only does the first part | 18:11 |
| xorAxAx | and pypy is not the tool to protect coders from creating deadlock-patterns | 18:11 |
| xorAxAx | Rhamphoryncus: i think its fairly simple. if all cpython opcodes just need to lock one object concurrently, implementing this should be nearly a no-brainer :) | 18:12 |
| xorAxAx | but i am not sure if this is the case | 18:12 |
| jacob22|home | xorAxAx: Not that I know how much there would be to gain in being able to multi-thread the interpreter. | 18:12 |
| Rhamphoryncus | xorAxAx: oh, did I mention it'd destroy performance? :) | 18:13 |
| xorAxAx | Rhamphoryncus: hmm, which performance? ;-) | 18:13 |
| Rhamphoryncus | CPython only touches the GIL once per *100* opcodes | 18:13 |
| xorAxAx | of course this is orthogonal to jitting | 18:13 |
| jacob22|home | xorAxAx: I suppose parallising the loading of modules would be the greatest advantage. | 18:14 |
| Rhamphoryncus | per-object locks would require touching them once per every function | 18:14 |
| xorAxAx | jacob22|home: umm | 18:14 |
| Rhamphoryncus | and at that point the locking overhead becomes the bottleneck | 18:14 |
| xorAxAx | jacob22|home: parallelising the interpreter means that it can benifit from multiple nodes/cpus | 18:14 |
| xorAxAx | why loading of modules? | 18:14 |
| Rhamphoryncus | SO much that you lose what you gain from having multiple CPUs | 18:15 |
| Rhamphoryncus | some algorithms break even with 4 CPUs. Many do not | 18:15 |
| xorAxAx | in the future, every system will have 4 cores :) | 18:15 |
| Rhamphoryncus | contrast that to lockless threading (which could be done with sandboxes). They're essentially free on a 1 CPU box, and it only gets better on a multiple CPU box | 18:15 |
| xorAxAx | Rhamphoryncus: or micro threads in erlang :) | 18:17 |
| Rhamphoryncus | xorAxAx: I'm talking hardware level here. Repeated locking kills performance, lockless algorithms do not | 18:18 |
| cfbolz | Rhamphoryncus: you really can do a lot of that with py.execnet already | 18:18 |
| Rhamphoryncus | If erlang uses locking (such as with a queue) then it may very well be slower than a single-threaded program | 18:18 |
| Rhamphoryncus | cfbolz: is that a pypy thingy? | 18:18 |
| xorAxAx | Rhamphoryncus: umm, you dont use global queues in erlang | 18:19 |
| cfbolz | Rhamphoryncus: no, it's part of the py-lib | 18:19 |
| xorAxAx | Rhamphoryncus: you simply queue messages for someone | 18:19 |
| Rhamphoryncus | xorAxAx: isn't tuplespace (isn't that what it's called?) a form of a global queue? | 18:19 |
| cfbolz | Rhamphoryncus: http://codespeak.net/py/current/doc/execnet.html | 18:19 |
| xorAxAx | Rhamphoryncus: just messages need to queued in erlang AFAIK, what else? | 18:19 |
| xorAxAx | Rhamphoryncus: yeah, right | 18:20 |
| xorAxAx | Rhamphoryncus: but they are thread-local, those tuplespaces | 18:20 |
| Rhamphoryncus | xorAxAx: umm, being global doesn't matter anyway. Queues in general require locks :) | 18:20 |
| Rhamphoryncus | eh? erlang doesn't have threads | 18:20 |
| Rhamphoryncus | it has "processes", which many-to-few map to OS threads | 18:21 |
| xorAxAx | i mean every micro thread has one, sorry | 18:21 |
| Rhamphoryncus | xorAxAx: I'm *sure* the micro threads communicate... | 18:21 |
| jacob22|home | xorAxAx: It is a matter of the code being executable in parallel at all. At what level you can express independence between execution chains. | 18:21 |
| xorAxAx | yeah, they queue messages | 18:21 |
| xorAxAx | jacob22|home: we are assuming separate threads here | 18:21 |
| xorAxAx | e.g. my windows has 63 processes and 571 threads currently :) | 18:22 |
| jacob22|home | xorAxAx: So, what parts of the interpreter can you profitably thread? | 18:22 |
| Rhamphoryncus | I actually doubt erlang does all that good for simple tasks. It scales very well and does a few other cool things, but it probably has a lot of overhead | 18:22 |
| xorAxAx | jacob22|home: it just needs to be run in one thread | 18:22 |
| xorAxAx | jacob22|home: but concurrently | 18:23 |
| xorAxAx | Rhamphoryncus: "if it scales well", it can just have constant overhead | 18:23 |
| Rhamphoryncus | hmm. Otoh, if the ordering semantics are loose enough then they could consolidate queue accesses and mitigate the locking cost | 18:23 |
| Rhamphoryncus | and since it's intended to be distributed it probably has loose ordering semantics | 18:24 |
| xorAxAx | yeah, the messages just need to match | 18:24 |
| xorAxAx | the signature or something like that | 18:24 |
| Rhamphoryncus | yeah, it's all pattern matching | 18:25 |
| jacob22|home | xorAxAx: Concurrently with what? | 18:25 |
| xorAxAx | jacob22|home: another interpreter thread | 18:25 |
| xorAxAx | just imagine you won in the lottery and have a machine with 4 cores. using python, you would just be able to use 25% of it | 18:26 |
| cfbolz | that's nonsense | 18:26 |
| cfbolz | just use four interpreters | 18:26 |
| xorAxAx | cfbolz: but then, you have the disadvantages of scaling via processes. bigger memory overhead or passing of data (via execnet?) | 18:27 |
| jacob22|home | xorAxAx: Running separate programs is trivial. OS:es know how to do do that. The interesting thing is to make a single program run faster using multiple CPU:s. | 18:28 |
| Rhamphoryncus | heh, there's still the old standbye.. if performance is really a problem you should rewrite the bottleneck in C ;) | 18:28 |
| xorAxAx | jacob22|home: yes | 18:28 |
| cfbolz | but if it is trivial, why then not just do it :-) | 18:28 |
| Rhamphoryncus | jacob22|home: and not go insane while doing it ;) | 18:28 |
| xorAxAx | hmm, are there any IPC libs for python optimised for local usage and large amount of data? | 18:28 |
| jacob22|home | xorAxAx: This is not done by running multiple interpreters. | 18:28 |
| xorAxAx | which in fact simplify Rhamphoryncus' model? | 18:29 |
| xorAxAx | jacob22|home: why not? lets assume that someone thought about 2 funcs being executed at the same time | 18:29 |
| xorAxAx | jacob22|home: then there need to be two threads at the OS level running those 2 funcs without global locks | 18:30 |
| xorAxAx | and those 2 threads need one interpreter each, dont they? | 18:31 |
| Rhamphoryncus | Anyway, I'm gonna head off to bed. G'night all :) | 18:31 |
| cfbolz | g'night :-) | 18:32 |
| jacob22|home | xorAxAx: Yes, and they need atomicity over multiple objects. We are back at my earlier objection. | 18:32 |
| xorAxAx | gn Rhamphoryncus | 18:32 |
| arigo | xorAxAx: Samuele points me to POSH (http://poshmodule.sourceforge.net/posh/html/), never tried, though | 18:32 |
| Rhamphoryncus (n=rhamph@unaffiliated/rhamphoryncus) left irc: "Support ISO 8601! http://www.cl.cam.ac.uk/~mgk25/iso-time.html" | 18:32 | |
| xorAxAx | hmm, discontinued since 2003 :) | 18:33 |
| cfbolz | indeed | 18:33 |
| xorAxAx | jacob22|home: yeah, i posed a question: is it possible to write implementations for every byte code word that just need to lock one object at the same time | 18:34 |
| xorAxAx | then you can use the simple locking idea from above | 18:34 |
| xorAxAx | arigo: and he somehow knows that it is the only one? | 18:35 |
| arigo | it seems so | 18:36 |
| jacob22|home | arigo: It also seems to invoke the overhead of one OS process per thread. That is fairly expensive. | 18:37 |
| xorAxAx | jacob22|home: yeah, processes are heavy weight | 18:37 |
| xorAxAx | jacob22|home: you choose threads or micro threads if you need many or few often and fast ... | 18:38 |
| arigo | jacob22|home: that was an answer to the question "is there a Python module for fast high-volume IPC", nothing more | 18:38 |
| jacob22|home | arigo: Ok. | 18:39 |
| xorAxAx | but IPC using python is memory intensive | 18:40 |
| ita (n=ita@kde/developer/tnagy) joined #pypy. | 18:42 | |
| ita | pypy not compiling - gc/gc.h not found - any idea ? | 18:42 |
| cfbolz | ita: yes, you need the boehm garbage collector installed | 18:43 |
| alextp | install libgc-dev | 18:43 |
| alextp | or something like it | 18:43 |
| ita | oh | 18:43 |
| ita | thanks | 18:43 |
| cfbolz | or you have to use reference counting | 18:43 |
| cfbolz | via the --gc=ref option | 18:43 |
| cfbolz | to translate_pypy | 18:43 |
| Action: ita retries with gc=ref | 18:50 | |
| cfbolz | gave up with boehm already? :-) | 18:50 |
| ita | no package for my distro | 18:51 |
| cfbolz | oh, I see | 18:51 |
| cfbolz | what do you use? | 18:51 |
| xorAxAx | good that windows users dont need to look for packages :) | 18:51 |
| ita | opensuse 10 - but i guess --gc=ref needs to be tested too | 18:51 |
| cfbolz | ita: opensuse has boehm | 18:52 |
| cfbolz | (I think) | 18:52 |
| alextp | it should have | 18:52 |
| ita | not on my dvd so i have given up | 18:52 |
| ita | still on rc1 | 18:52 |
| cfbolz | but 9.3 had an _extremely_ old version, maybe that did not change | 18:52 |
| cfbolz | (so old as to be unusable with pypy) | 18:52 |
| cfbolz | ita: the problem with refcounting is that we don't have any cycle detection yet. so cyclic garbage will never be reclaimed | 18:57 |
| ita | uh-oh | 18:57 |
| ita | i should not make cyclic structures then ? | 18:57 |
| ita | :) | 18:57 |
| cfbolz | indeed | 18:57 |
| cfbolz | :-) | 18:57 |
| ita | i only have trees | 18:57 |
| cfbolz | pypy is not really production ready anyway | 18:58 |
| ita | i have seen - no threading for example | 18:58 |
| ita | cpickle not working in the regression tests, only pickle | 18:58 |
| xorAxAx | it has threading. it just blocks too often | 18:58 |
| cfbolz | xorAxAx: not right now, there is a bug that breaks it | 18:58 |
| xorAxAx | cfbolz: ah, ok | 18:58 |
| cfbolz | ita: "c"pickle does of course not make sense with pypy :-) | 18:59 |
| ita | will stat-ing files be faster with pypy ? :) | 18:59 |
| cfbolz | at the moment nothing is faster with pypy | 18:59 |
| xorAxAx | 10 times slower i guess :) | 18:59 |
| arigo | ita: cPickle actually works. The 5 reported failures test an undocumented interface of Python's cPickle... | 19:07 |
| arigo | (which is setting 'fast=1' on the pickler object -- who knows about it?) | 19:07 |
| cfbolz | arigo: but for pypy pickle = cpickle, right? | 19:08 |
| arigo | almost, yes | 19:08 |
| cfbolz | almost? | 19:08 |
| arigo | look at pypy/lib/cPickle.py :-) | 19:09 |
| arigo | it says "one-liner implementation of cPickle", but it's 40 lines actually | 19:09 |
| cfbolz | :-) | 19:09 |
| ita | some debug info .. [rtyper:WARNING] PBC <pypy.objspace.std.stdtypedef.StdTypeDef instance at 0x4072bd0c> has no attribute 'memofield_0' | 19:12 |
| cfbolz | that's a known issue | 19:12 |
| cfbolz | (although I don't know what the actual problem is :-) | 19:12 |
| arigo | indeed. We looked it up but I don't remember | 19:14 |
| arigo | it was fine, though | 19:15 |
| cfbolz | "the mystery error" | 19:15 |
| arigo | "warning" :-) | 19:17 |
| cfbolz | oh, yes. the mystery _warning_ :-) | 19:18 |
| Nick change: ita -> gruik | 19:47 | |
| gruik | anything to do for ImportError: No module named pygame ? | 20:34 |
| alextp | install pygame | 20:34 |
| gruik | hehe ok :) | 20:34 |
| alextp | :) | 20:34 |
| gruik (n=ita@84.4.70.25) left irc: Remote closed the connection | 20:43 | |
| bima (n=loubna_g@adsl-138-193-192-81.adsl.iam.net.ma) joined #pypy. | 20:49 | |
| arigo (n=arigo@c-3c8b70d5.022-54-67626719.cust.bredbandsbolaget.se) left irc: "[x]chat" | 21:34 | |
| alextp (n=top@201008227201.user.veloxzone.com.br) left #pypy. | 21:50 | |
| xorAxAx | hmm, posh uses fork. | 22:08 |
| xorAxAx | i wonder why there are no other IPC toolkits for python | 22:08 |
| cfbolz | because all people interested in that are like discussion more than implementing stuff? | 22:09 |
| xorAxAx | no | 22:10 |
| xorAxAx | because they use e.g. jython | 22:10 |
| cfbolz | why is the topic than raised so often? | 22:10 |
| xorAxAx | and because it is not trivial | 22:10 |
| xorAxAx | cfbolz: because you want to use more than one processor node and dont work around strange limits like an inner xmlrpc system (easy to implement, slow for IPC) | 22:11 |
| xorAxAx | cfbolz: assuming that you are going to implement it using multiple processes | 22:12 |
| cfbolz | sorry, my problem with all these discussions is that I never personally found a case where all that stuff would be useful for me | 22:12 |
| xorAxAx | i just have one multiprocessor machine - a very old sparcstation II. so i wont benefit from it | 22:12 |
| xorAxAx | umm, 20, not II | 22:12 |
| cfbolz | so I might be ignorant | 22:13 |
| yota (n=seb@home.nmeos.net) joined #pypy. | 22:13 | |
| xorAxAx | cfbolz: imagine you have a program which is cpu bound | 22:13 |
| xorAxAx | cfbolz: which has to work with data | 22:13 |
| cfbolz | sure, I have loads of them | 22:13 |
| xorAxAx | cfbolz: and you have two processor nodes | 22:13 |
| xorAxAx | now all is written i npython | 22:13 |
| cfbolz | I _do_ have that situation often | 22:14 |
| xorAxAx | python has the GIL like any other scripting language that is comparable | 22:14 |
| cfbolz | but it could always be solved by IPC | 22:14 |
| xorAxAx | i _am_ talking about IPC | 22:14 |
| xorAxAx | so how do you do that IPC? | 22:14 |
| xorAxAx | posh is about IPC | 22:14 |
| cfbolz | py.execnet suits my needs. pipes do it most of the time, too | 22:14 |
| xorAxAx | pipes ... | 22:15 |
| cfbolz | sure | 22:15 |
| cfbolz | py.execnet is just a clever way to wrap pipes, btw | 22:15 |
| xorAxAx | does execnet LPC/RPC? | 22:16 |
| cfbolz | what is that? | 22:16 |
| xorAxAx | i.e. can you call procedures with it? | 22:16 |
| xorAxAx | the name suggests it | 22:16 |
| cfbolz | yes, you can | 22:16 |
| xorAxAx | but you will realise that it will be the limiting factor in apps with much data | 22:16 |
| cfbolz | _some_ apps with much data, I agree | 22:17 |
| cfbolz | but not all apps with much data have that problem | 22:17 |
| xorAxAx | esp. if you are passing non-string data | 22:19 |
| cfbolz | most of the time datapassing was not the limiting factor for me | 22:19 |
| cfbolz | consider raytracing a movie, for example | 22:19 |
| cfbolz | very nicely clusterizable | 22:20 |
| xorAxAx | if the raytracer is written in python and you want to shift the movie into a codec (written in python %-) you have a lot of bandwith | 22:20 |
| xorAxAx | serialising that into a pipe is more expensive than passing around a pointer to a ring buffer | 22:20 |
| cfbolz | ok, but you are not going to divide into raytracer/codec | 22:21 |
| cfbolz | but rather into frames | 22:21 |
| xorAxAx | which gives you many stills that have to go to the codec | 22:21 |
| cfbolz | yes, but the codec is usually cheap compared to raytracing | 22:22 |
| xorAxAx | hmm, yeah. do you know the 500 bytes raytracer written in postscript? | 22:22 |
| cfbolz | sure | 22:22 |
| xorAxAx | it needed 2 hours on my printer | 22:22 |
| cfbolz | I never had the patience to wait. just did it with qhostscript | 22:22 |
| xorAxAx | :-) | 22:22 |
| xorAxAx | while we are at parallelisation: a backend for the cell processor would be nice | 22:24 |
| cfbolz | not easy | 22:24 |
| xorAxAx | why? | 22:24 |
| cfbolz | well, depends how you want to access the multiple cores | 22:25 |
| cfbolz | but doing it automatically is hard | 22:25 |
| xorAxAx | you just define entry points that are getting translated separately | 22:25 |
| cfbolz | "just"? | 22:25 |
| xorAxAx | :-) | 22:25 |
| cfbolz | :-) | 22:25 |
| cfbolz | let us just write the JIT first | 22:26 |
| xorAxAx | umm, where is the difficulty having an entry point that just calls a few funcs (those that should go to the cores) | 22:26 |
| xorAxAx | of course you need a lot of glue to let it work out of the box suitable for developing | 22:27 |
| cfbolz | indeed | 22:27 |
| xorAxAx | but its not interesting anyway as long as nobody has such a playstation | 22:27 |
| cfbolz | right. maybe we can get IBM to sponsor us :-) | 22:27 |
| xorAxAx | yeah :) | 22:28 |
| xorAxAx | "rpython is easier than pseudo code. let your young consumers code shaders and transformers in simple lines of python" | 22:29 |
| cfbolz | would be cool, actually. I would love to be able to do that | 22:30 |
| cfbolz | although I think there was a python frontend to some graphic card shading language | 22:30 |
| xorAxAx | hmm, there is blender :) | 22:31 |
| xorAxAx | and http://graphics.cs.lth.se/pyfx/ | 22:32 |
| xorAxAx | hmm, looks nice | 22:32 |
| cfbolz | the blender->python binding useable, but a bit strange | 22:32 |
| xorAxAx | why? | 22:33 |
| cfbolz | it's a bit, well, unpythonic :-) | 22:33 |
| xorAxAx | the psp binding is very ... verbose | 22:33 |
| cfbolz | yow, pyfx looks cool | 22:33 |
| cfbolz | psp? | 22:34 |
| xorAxAx | they have 30 -50 kwargs for many methods/classes :) | 22:34 |
| xorAxAx | paint shop pro | 22:34 |
| cfbolz | uah | 22:34 |
| xorAxAx | but they have a macro recorder | 22:34 |
| cfbolz | that records python? | 22:34 |
| xorAxAx | yes | 22:34 |
| cfbolz | that is cool | 22:34 |
| xorAxAx | yes | 22:34 |
| xorAxAx | better than a VBA recorder :) | 22:34 |
| cfbolz | I learned what little I know of visual basic by looking a recorded code | 22:34 |
| xorAxAx | yeah, there code generator is really advanced | 22:35 |
| xorAxAx | s/there/their/ | 22:35 |
| xorAxAx | and its using keyworded args very often | 22:35 |
| xorAxAx | i have never seen that in plain vb code such often | 22:36 |
| cfbolz | gimps python binding is quite ok too, I think | 22:37 |
| stakkars | howdy | 22:38 |
| cfbolz | hi christian! | 22:38 |
| stakkars | nice weather here in Beverly Hills | 22:39 |
| xorAxAx | hi stakkars | 22:39 |
| xorAxAx | vacations? :-) | 22:39 |
| stakkars | xorAxAx: Hi! Thanks for helping me out with testing | 22:39 |
| xorAxAx | no problem. i found 15 bugs as you might have read ;-) | 22:39 |
| stakkars | no, paid work, some Python optimization | 22:40 |
| xorAxAx | which could be tracked down to missing quotes | 22:40 |
| xorAxAx | stakkars: ah, ok | 22:40 |
| stakkars | (didn't read yet, was veery busy) | 22:40 |
| stakkars | maybe this is becoming the first tiny commercial PyPy application. | 22:40 |
| xorAxAx | oh, why didnt they choose cpython? | 22:41 |
| xorAxAx | because it yields a ZeroDivisionError in their timer-calibrator? ;-) | 22:41 |
| stakkars | they do use CPython 2.4. They are processing gigs of data with this, and some very basic | 22:41 |
| stakkars | stuff like parsing a line of text and extracting some fields takes too much time. | 22:41 |
| stakkars | I can probably isolate this and generate a small RPython extension module for it. | 22:42 |
| cfbolz | heh | 22:42 |
| xorAxAx | ah, so you want to use rpython like it was not meant to be used? ;-)I | 22:42 |
| Action: cfbolz hits xorAxAx | 22:42 | |
| cfbolz | :-) | 22:42 |
| xorAxAx | ouch :) | 22:42 |
| stakkars | yes. Inever felt too hapy with what I was supposed to think is "meant". If I can use anything right now, I'l do it. :-) | 22:43 |
| xorAxAx | thats the pythonic pragmatic approach | 22:43 |
| cfbolz | indeed | 22:43 |
| stakkars | it will take long until we can use PyPy to optimize stuff like that. And I want to keep them away from Pyrex et. al. | 22:43 |
| stakkars | they are Python newbies and write a quite well optimized system for processing their data. | 22:44 |
| stakkars | but they can't get this kind of code fast enough: | 22:44 |
| xorAxAx | so you will hit a problem in the documentation phase - at the point where you explain how to maintain and extend the code, nor? | 22:45 |
| stakkars | code = line[0]; amount = int(line[1:36]; name = line[36:50]; isBid = line[50] == 'B' | 22:45 |
| xorAxAx | ebay? | 22:45 |
| xorAxAx | :-) | 22:45 |
| cfbolz | that indeed looks like it could be done nicely with rpython | 22:45 |
| stakkars | yes, this will be just an intermediate approach to get them some speed. | 22:45 |
| stakkars | absolutely, and this is the stuff where we already create incredible fast code. | 22:46 |
| stakkars | I'm just testing whether I can get at the data, easily. I think we do support returning a tuple. | 22:46 |
| stakkars | EWD, stock trading | 22:46 |
| stakkars | sory EWT. | 22:46 |
| cfbolz | I am not sure | 22:46 |
| xorAxAx | rpython seems to made for string processing :) | 22:46 |
| xorAxAx | s/to m/to be m/ | 22:47 |
| stakkars | I saw one piece of code in the tests where we return a tuple of numbers and check it in the CPython part. | 22:47 |
| stakkars | at the same time, it was said that we cannot produce a list because it will be an RList. | 22:48 |
| cfbolz | tuples work | 22:48 |
| stakkars | however, I'd be happy to find a way to use PyPy here, although it is immature. *** GREAT *** | 22:48 |
| cfbolz | sure, list is a bad idea, since the types have to match | 22:49 |
| stakkars | the idea is to create new sponsors by this. And this is a good one, the really need speed, and they love Python. Their stuff is almost all Java, but they wont to be more flexible. | 22:50 |
| cfbolz | and they are switching over? | 22:50 |
| jacob22|home | stakkars: How long will you be in the US? | 22:50 |
| stakkars | if I can work for them from time to time, this is just good for the project. | 22:50 |
| stakkars | It is planned that I leave on the 14th in the evening. But the ticket can be extended. | 22:51 |
| jacob22|home | stakkars: Yes, it sounds quite good. | 22:51 |
| stakkars | jacob22|home: I thought of our conversation when I negotiated this contract. I was apporached by head-hunters which had read conversations between Armin and me. :-) PyPy rules! | 22:52 |
| jacob22|home | stakkars: I hope you can do some Pypy stuff while you are there. I'll have my deliverables ready for review in a couple of days and we really need your timesheets. | 22:52 |
| jacob22|home | stakkars: Yes, I think Pypy is good for everyone involved. | 22:53 |
| stakkars | yes I planned to have time for reviewing. And for the latter: this is my spare day, and I'm going to finish the time sheet never-ending-story today, not programming, just a little sight-seeing and that. | 22:53 |
| jacob22|home | stakkars: Great! | 22:54 |
| xorAxAx | stakkars: what kind of "conversation" do those head hunters monitor? | 22:54 |
| xorAxAx | public ML? | 22:54 |
| stakkars | just activites on mailing lists. | 22:55 |
| xorAxAx | ok | 22:55 |
| jacob22|home | stakkars: I can recommend the Getty museum, and I have heard that the La Brea (tarpits with dinosaur fossiles) is really something to see. | 22:55 |
| stakkars | yes. And I got a phone call from a nice sounding woman, and then I got contacts to the core people of EWT, and finally I got the best rate I ever had. :-)) | 22:55 |
| xorAxAx | %-) | 22:56 |
| stakkars | jacob22|home: hey, this is great, thanks! It is good to have you around. How about "Jacob's guide to the Galaxy"? :) | 22:56 |
| jacob22|home | stakkars: Nah, I just hand out tips to people I like. ;-) | 22:58 |
| cfbolz | hehe | 22:58 |
| jacob22|home | I even managed to get around in LA by bus. That was difficult though. | 22:59 |
| bima (n=loubna_g@adsl-138-193-192-81.adsl.iam.net.ma) left #pypy. | 22:59 | |
| xorAxAx | jacob22|home: why is that? | 22:59 |
| stakkars | I got a bicycle, will see if this is enought to get at the places. | 23:00 |
| cfbolz | the USA are not always friendly to public transport. depends on the city, though | 23:01 |
| stakkars | public transportation in LA is almost only buses. They got three tram lines, all not in my area. | 23:01 |
| stakkars | compared to SF or Washington DC, this is nothing. | 23:02 |
| cfbolz | not to talk about New York | 23:02 |
| jacob22|home | stakkars: It should be. Stay off the main streets though. You are usually allowed to ride bikes on the sidewalks in US cities. | 23:02 |
| stakkars | but the citiy is incedibly nice. I can just step out of the door and start photo shooting. | 23:02 |
| stakkars | I've never been to New York but for changing planes. Will happen some time. | 23:03 |
| cfbolz | you should! it's a wonderful city | 23:03 |
| jacob22|home | stakkars: You have missed very little. Tall buildings and snotty people. | 23:04 |
| cfbolz | jacob22|home: careful, I have relatives there :-) | 23:04 |
| stakkars | yes I was told this in the introduction to EWT and how I should get along. | 23:05 |
| stakkars | they are from New York, partially. and their working hours are New York time, to be in sync with trading. | 23:05 |
| cfbolz | so you have to get up early? | 23:06 |
| jacob22|home | stakkars: Ahh. Refugees. ;-P | 23:06 |
| stakkars | it was my first day, yesterday, yet-lagged and tired, but enough to let them conclude "we're excited" in the evening. | 23:06 |
| stakkars | yes, I'm starting between 7:30 and 9:00 | 23:06 |
| stakkars | jacob22|home: yes :-) | 23:07 |
| jacob22|home | stakkars: That's good. You won't be so awfully jetlagged when you get home. Going to bed late in the evening California time makes for very bad sleeping patterns when you get home. | 23:08 |
| stakkars | jacob22|home. yeah. Although I'm working 10+ hours. | 23:09 |
| stakkars | jacob22|home: I was thinking of hiring Richard Emslie part-time for my company and let him do some paid PyPy work, similar to what Holger does with Carl. Would this be possible, or is it a problem, time-sheet wise? | 23:10 |
| jacob22|home | I'm in the middle of porting all my CDs to a hard disk. I just did the 100th one. | 23:10 |
| cfbolz | stakkars: might be a problem, because richard is becoming a physical person | 23:10 |
| jacob22|home | stakkars: There may be a problem in that he is an NP member of the project. | 23:11 |
| xorAxAx | cfbolz: you are virtual? :-) | 23:11 |
| cfbolz | yes | 23:11 |
| jacob22|home | The PO is investigating. | 23:11 |
| xorAxAx | cfbolz: so you are just a chat bot? | 23:11 |
| cfbolz | let's not go down that road, there are far too many bots in the python community already | 23:12 |
| xorAxAx | but where is the different between a virtual and a physical guy here? | 23:12 |
| cfbolz | the physical person gets his sprint travel refunded | 23:13 |
| cfbolz | by the EU | 23:13 |
| stakkars | ah, Richard is getting a role in PyPy, already, which contradicts to work for me? | 23:13 |
| cfbolz | stakkars: indeed, he will become a physical person | 23:13 |
| cfbolz | and it's not clear whether both at the same time works | 23:13 |
| stakkars | eeek! I need somebody. Maybe I can buy a piece of Carl the Virtual | 23:14 |
| xorAxAx | does he support LPAR already? | 23:14 |
| cfbolz | stakkars: don't worry, it might still work out | 23:14 |
| jacob22|home | stakkars: It may be that he can't work for you at all. This is unlikely. It may be that he can work for you if he resigns as a NP in the project. It may be that he can stay on as a NP, as long as he doesn't get any reimbursement directly from the EU while he works for you. We don't know yet. | 23:15 |
| jacob22|home | stakkars: Holger has the same issue with hiring Eric. | 23:15 |
| stakkars | ok. But you what I'm after: If I officially do work for other companies, I'd like to have somkebody jump in for my PyPy work, not missing funding. | 23:16 |
| stakkars | s/you/you know/ | 23:16 |
| jacob22|home | stakkars: No problem, as long as the one jumping in is an official employee of your company. No contractors. | 23:17 |
| cfbolz | stakkars: exactly what holger does as well | 23:17 |
| stakkars | that's what I'm trying to do. Just someone tell me at some point whom I can employ. | 23:17 |
| stakkars | it is not so easy because it has to be a valid PyPy developer | 23:18 |
| stakkars | who is also accepted by the group | 23:18 |
| jacob22|home | stakkars: I'll keep an eye on what comes from the PO and I'll make sure you get the info. | 23:18 |
| stakkars | jacob22|home: thank you so much! | 23:19 |
| ericvrp (n=ericvrp@ericvrp.demon.nl) joined #pypy. | 23:39 | |
| jacob22|home | ericvrp: Hi! Javascript backend looks good. | 23:42 |
| ericvrp | thank you | 23:43 |
| ericvrp | I have not been very busy with it lately. My last checkin (where testing against browsers works) is what kept me back for a while | 23:44 |
| xorAxAx | + #note: 0 second did not work probably because the GET would | 23:45 |
| xorAxAx | +<meta http-equiv="refresh" content="0"> | 23:45 |
| xorAxAx | hmm :) | 23:45 |
| xorAxAx | maybe a 1 should go there? | 23:45 |
| cfbolz | ericvrp: are the tests expected to pass? | 23:47 |
| cfbolz | all of them, I mean? | 23:48 |
| cfbolz | some do | 23:48 |
| ericvrp | oops. that comment should no longer be there. The problem was something else... firefox did a get /favicon.ico | 23:48 |
| ericvrp | no. on my system 90 pass, there rest don't. | 23:48 |
| cfbolz | ok | 23:48 |
| cfbolz | it worked after fixing the path | 23:48 |
| cfbolz | which is highly OS dependant, I fear | 23:48 |
| cfbolz | and you really should put a note somewhere that your browser gets killed :-) | 23:49 |
| ericvrp | test/test_class.py[16] ....FFF.FF..F.F. | 23:49 |
| cfbolz | same pattern for me | 23:49 |
| ericvrp | maybe :/ | 23:49 |
| cfbolz | not so bad :-) | 23:50 |
| ericvrp | maybe a sys.platform test? | 23:50 |
| cfbolz | firefox also supports nice options where you can open the stuff in a new tab, btw | 23:50 |
| cfbolz | maybe, yet | 23:50 |
| cfbolz | yes | 23:50 |
| cfbolz | is this related to the browser-based js tests that guido is working on? | 23:51 |
| ericvrp | yes | 23:51 |
| cfbolz | or just a coincidence | 23:51 |
| ericvrp | no, I visited him last week and this is based on his code. | 23:51 |
| cfbolz | very cool | 23:51 |
| cfbolz | 89 passed | 23:52 |
| ericvrp | The differnce is mainly that he uses an external javascript unittest framework and does all the tests at once, while I run test-for-test with some simple html to fetch the next test. | 23:52 |
| ericvrp | It really need to be cleaned up! | 23:52 |
| cfbolz | that's ok | 23:53 |
| ericvrp | very odd, 90 pass here. Sometimes spidermonkey is not reproducable too. Maybe some undefined js variable? ah well... | 23:54 |
| cfbolz | no idea. it scrolled away | 23:54 |
| xorAxAx | argh, it doesnt like spaces in the filename | 23:54 |
| ericvrp | Probably quotes aroung the url would fix that easilyt | 23:54 |
| xorAxAx | ericvrp: do you know the webbrowser module? maybe it can help here | 23:55 |
| cfbolz | indeed, that's an idea | 23:55 |
| ericvrp | * googling | 23:55 |
| cfbolz | it's part of the stdlib | 23:55 |
| xorAxAx | :-) | 23:56 |
| ericvrp | this is nice. more discussion now already then all the time with llvm. I LOVE javascript :) | 23:56 |
| cfbolz | that is not true | 23:56 |
| ericvrp | it the beginning no | 23:56 |
| xorAxAx | what do you mean, ericvrp? | 23:56 |
| cfbolz | quite a lot of the llvm discussion predates you :-) | 23:56 |
| ericvrp | yes | 23:56 |
| ericvrp | but lately there has not been alot of interest was there? | 23:57 |
| cfbolz | no, now it works more or less :-) | 23:57 |
| cfbolz | a solved problem is uninteresing | 23:57 |
| cfbolz | (an unsolved one two, of course) | 23:57 |
| ericvrp | yes, and it proves pypy is plexable, but that's it. | 23:58 |
| xorAxAx | are there llvm vs. c benchmarks? | 23:58 |
| cfbolz | ericvrp: oh come on. don't start with this again. it is a worthwhile backend to have | 23:58 |
| ericvrp | but I'm not complaining, I just think it is nice that you people are looking at genjs now | 23:58 |
| ericvrp | xorAxAx: the speed is currently comparable | 23:58 |
| cfbolz | genjs will certainly show that javascript is _not_ flexible :-) | 23:59 |
| xorAxAx | and not fast :) | 23:59 |
| --- Sun Nov 6 2005 | 00:00 | |
Generated by irclog2html.py 2.9.2 by Marius Gedminas - find it at mg.pov.lt!