Ok, after going through serialization code, it's clear that default
implementation doesn't support serializing function code, but only its
name. For example, here's relevant section from
`deserialize(::SerializationState, ::Function)`:
mod = deserialize(s)::Module
name = deserialize(s)::Symbol
if !isdefined(mod,name)
return (args...)->error("function $name not defined on process
$(myid())")
end
This doesn't fit my needs (essentially, semantics of Spark), and I guess
there's no existing solution for full function serialization. Thus I'm
going to write new solution for this.
So far the best idea I have is to get function's AST and recursively
serialize it, catching calls to the other non-Base function and any bound
variables. But this looks quite complicated. Is there better / easier way
to get portable function's representation?
On Monday, August 10, 2015 at 11:48:55 PM UTC+3, Andrei Zh wrote:
>
> Yes, I incorrectly assumed `serialize` / `deserialize` use JLD format. But
> anyway, even when I saved the function into "example.jls" or even plain
> byte array (using IOBuffer and `takebuf_array`), nothing changed. Am I
> missing something obvious?
>
>
> On Monday, August 10, 2015 at 11:40:03 PM UTC+3, Tim Holy wrote:
>>
>> On Monday, August 10, 2015 01:13:15 PM Tony Kelman wrote:
>> > Should
>> > probably use some different extension for that, .jls or something, to
>> avoid
>> > confusion.
>>
>> Yes. That has been sufficiently confusing in the past, we even cover this
>> here:
>>
>> https://github.com/JuliaLang/JLD.jl#saving-and-loading-variables-in-julia-data-format-jld
>>
>>
>> --Tim
>>
>> >
>> > On Monday, August 10, 2015 at 12:45:35 PM UTC-7, Stefan Karpinski
>> wrote:
>> > > JLD doesn't support serializing functions but Julia itself does.
>> > >
>> > > On Mon, Aug 10, 2015 at 3:43 PM, Andrei Zh <[email protected]
>> > >
>> > > <javascript:>> wrote:
>> > >> I'm afraid it's not quite true, and I found simple way to show it.
>> In the
>> > >> next code snippet I define function `f` and serialize it to a file:
>> > >>
>> > >> julia> f(x) = x + 1
>> > >> f (generic function with 1 method)
>> > >>
>> > >> julia> f(5)
>> > >> 6
>> > >>
>> > >> julia> open("example.jld", "w") do io serialize(io, f) end
>> > >>
>> > >>
>> > >> Then I close Julia REPL and in a new session try to load and use
>> this
>> > >> function:
>> > >>
>> > >> julia> f2 = open("example.jld") do io deserialize(io) end
>> > >> (anonymous function)
>> > >>
>> > >> julia> f2(5)
>> > >> ERROR: function f not defined on process 1
>> > >>
>> > >> in error at error.jl:21
>> > >> in anonymous at serialize.jl:398
>> > >>
>> > >> So deserialized function still refers to the old definition, which
>> is not
>> > >> available in this new session.
>> > >>
>> > >> Is there any better way to serialize a function and run it on an
>> > >> unrelated Julia process?
>> > >>
>> > >> On Monday, August 10, 2015 at 2:33:11 PM UTC+3, Jeff Waller wrote:
>> > >>>> My question is: does Julia's serialization produce completely
>> > >>>> self-containing code that can be run on workers? In other words,
>> is it
>> > >>>> possible to send serialized function over network to another host
>> /
>> > >>>> Julia
>> > >>>> process and applied there without any additional information from
>> the
>> > >>>> first
>> > >>>> process?
>> > >>>>
>> > >>>> I made some tests on a single machine, and when I defined function
>> > >>>> without `@everywhere`, worker failed with a message "function
>> myfunc
>> > >>>> not
>> > >>>> defined on process 1". With `@everywhere`, my code worked, but
>> will it
>> > >>>> work
>> > >>>> on multiple hosts with essentially independent Julia processes?
>> > >>>
>> > >>> According to Jey here
>> > >>> <
>> https://groups.google.com/forum/#!searchin/julia-users/jey/julia-users/
>> > >>> bolLGcSCrs0/fGGVLgNhI2YJ>, Base.serialize does what we want; it's
>> > >>> contained in serialize.jl
>> > >>> <https://github.com/JuliaLang/julia/blob/master/base/serialize.jl>
>>
>>
