tag:blogger.com,1999:blog-217898742024-03-07T02:54:27.130-07:00one racketeerRyan Culpepper
on macros, functional programming, and programming language researchRyan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-21789874.post-88517469188695447712013-06-05T14:23:00.001-06:002013-06-05T14:23:27.265-06:00define vs attach<p>Racket’s macro system gives macros two ways of associating
<span style="font-style: italic">static</span> (or <span style="font-style: italic">compile-time</span>) information with a particular
name. One way is to <span style="font-style: italic">define</span> the name as the static information;
the other is to <span style="font-style: italic">attach</span> the information to the already-defined
name.</p><a href="http://macrologist.blogspot.com/2013/06/define-vs-attach.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-69320773393133775642012-03-16T08:10:00.000-06:002012-03-16T08:10:12.965-06:00avoid flushing your wire protocol's performance down the pipes<p>When implementing a wire protocol, one occasionally needs to know something
about the wires. This blog post is the story of how the placement of a call to
<span class="RktSym"><a href="http://doc.racket-lang.org/reference/port-buffers.html#(def._((quote._~23~25kernel)._flush-output))" class="RktValLink" pltdoc="x">flush-output</a></span> caused a <span style="font-style: italic">factor-of-20</span> variation in performance.</p><a href="http://macrologist.blogspot.com/2012/03/avoid-flushing-your-wire-protocols.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-36715090814448619372012-02-09T12:41:00.001-07:002012-02-09T12:41:36.104-07:00unprepared queries vs statement caching<p>Racket’s <a href="http://pre.racket-lang.org/docs/html/db/index.html" class="RktModLink" pltdoc="x"><span class="RktSym">db</span></a> library supports parameterized queries
where the SQL is given as a string (that is, <a name="(tech._unprepared)"></a><span style="font-style: italic">unprepared</span>):</p><blockquote class="SCodeFlow"></blockquote><p><span class="RktPn">(</span><span class="RktSym"><a href="http://pre.racket-lang.org/docs/html/db/query-api.html#(def._((lib._db%2Fbase..rkt)._query-list))" class="RktValLink" pltdoc="x">query-list</a></span><span class="hspace"> </span><span class="RktSym">c</span><span class="hspace"> </span><span class="RktVal">"SELECT field1 FROM table WHERE field2 = ?"</span><span class="hspace"> </span><span class="RktVal">17</span><span class="RktPn">)</span></p><p>This is handled by an implicit call to <span class="RktSym"><a href="http://pre.racket-lang.org/docs/html/db/query-api.html#(def._((lib._db%2Fbase..rkt)._prepare))" class="RktValLink" pltdoc="x">prepare</a></span>, which turns
the string into a prepared statement; the prepared statement is then
executed with the arguments.</p><p>The problem with this approach is that it involves two trips to the
server: one to prepare and one to execute. One trip would be
better. In this post I discuss two techniques for eliminating
extraneous server trips. I’ve recently added one of them to Racket;
the other turns out to be problematic.</p><a href="http://macrologist.blogspot.com/2012/02/unprepared-queries-vs-statement-caching.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-12683723817080550462011-10-31T07:00:00.000-06:002011-11-13T11:22:21.953-07:00in praise of PostgreSQL arrays<p>I just added support for
<a href="http://www.postgresql.org/docs/current/static/arrays.html">PostgreSQL
arrays</a> to the <a href="http://pre.racket-lang.org/docs/html/db/index.html" class="RktModLink" pltdoc="x"><span class="RktSym">db</span></a> library. While there are some uses
of arrays that are iffy from a database design standpoint, there’s one
use that weighs overwhelmingly in their favor: avoiding dynamic
generation of SQL <span class="stt">IN</span> comparisons.</p>
<a href="http://macrologist.blogspot.com/2011/10/in-praise-of-postgresql-arrays.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com2tag:blogger.com,1999:blog-21789874.post-83826159131102445462011-10-27T00:46:00.000-06:002011-10-27T00:46:58.143-06:00the Soylent Green Theory of Presentations<p>It’s
people.... <a href="http://www.youtube.com/watch?v=8Sp-VFBbjpE"><span style="font-style: italic">The
audience is made of people!</span></a></p>
<a href="http://macrologist.blogspot.com/2011/10/soylent-green-theory-of-presentations.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-41254657915507275472011-10-16T23:28:00.000-06:002011-10-16T23:28:03.299-06:00lazy module loading<p>The Racket module system is good at managing dependencies. When you
<span class="RktSym"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket%2Fprivate%2Fbase..rkt)._require))" class="RktStxLink" pltdoc="x">require</a></span> a module, you ensure that that module is initialized
before your code runs, and when the other module changes, the compiler
will notice and recompile your module too. Racket even stratifies
dependencies according to <a href="http://docs.racket-lang.org/guide/stx-phases.html" pltdoc="x">phase
levels</a> so you can use some modules in your macro implementations and
other modules in your run-time code and the expander/compiler/linker
knows
<a href="http://www.cs.utah.edu/plt/publications/macromod.pdf">what
you want when</a>. It keeps track and makes sure that everything is
loaded and available when it’s supposed to be.</p><p>But sometimes you want to manage dependencies yourself. This post
is about how to <span style="font-style: italic">lazily load</span> the implementations of functions
and—<wbr>with a bit of care—<wbr>even macros.</p>
<a href="http://macrologist.blogspot.com/2011/10/lazy-module-loading.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com2tag:blogger.com,1999:blog-21789874.post-91076793597777619362011-09-27T04:03:00.001-06:002011-09-28T02:01:40.170-06:00definitions vs enclosing binding forms<p>There are two kinds of binding forms in Racket: definitions and
enclosing binding forms. The scope of a binding introduced by an
enclosing binding form is entirely evident: it’s one (or more) of the
form’s sub-terms. For example, in</p><blockquote class="SCodeFlow"><p><span class="RktPn">(</span><span class="RktSym"><a href="http://docs.racket-lang.org/reference/lambda.html#(form._((lib._racket%2Fprivate%2Fbase..rkt)._lambda))" class="RktStxLink" pltdoc="x">lambda</a></span><span class="hspace"> </span><span class="RktPn">(</span><span class="RktVar">var</span><span class="hspace"> </span><span class="RktSym"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket%2Fprivate%2Fstxcase-scheme..rkt)._......))" class="RktStxLink" pltdoc="x">...</a></span><span class="RktPn">)</span><span class="hspace"> </span><span class="RktVar">body</span><span class="RktPn">)</span></p></blockquote><p>the scope of the <span class="RktVar">var</span> bindings is <span class="RktVar">body</span>. In
contrast, the scope of a definition is determined by its context: the
enclosing <span class="RktSym"><a href="http://docs.racket-lang.org/reference/lambda.html#(form._((lib._racket%2Fprivate%2Fbase..rkt)._lambda))" class="RktStxLink" pltdoc="x">lambda</a></span> body, for example, or the enclosing
module—<wbr>except that scope is too simple a term for how bindings work
in such contexts. Enclosing binding forms are simpler and cleaner but
weaker; definition forms are more powerful, but have a more
complicated binding structure. Definitions also have the pleasant
property of reducing rightward code drift.</p>
<a href="http://macrologist.blogspot.com/2011/09/definitions-vs-enclosing-binding-forms.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-26362670939881997342011-09-09T02:49:00.002-06:002011-09-28T03:51:15.511-06:00syntax-parse and literals<p>In
<a href="http://macrologist.blogspot.com/2011/09/macros-and-literals.html">my
last post</a>, I talked about macros and referential auxiliary
identifiers—<wbr>what we usually call a macro’s “literals.” Scheme
macro systems only get it half right, though, because while they
compare identifiers using referential equality (i.e., using the
<span class="RktSym"><a href="http://docs.racket-lang.org/reference/stxcmp.html#(def._((quote._~23~25kernel)._free-identifier~3d~3f))" class="RktValLink" pltdoc="x">free-identifier=?</a></span> predicate), they allow literals to refer to
nonexistent bindings. While the comparison is well-defined via the
definition of <span class="RktSym"><a href="http://docs.racket-lang.org/reference/stxcmp.html#(def._((quote._~23~25kernel)._free-identifier~3d~3f))" class="RktValLink" pltdoc="x">free-identifier=?</a></span>, at a higher level the idea
is nonsensical.</p><p>In contrast, <span class="RktSym"><a href="http://docs.racket-lang.org/syntax/Parsing_Syntax.html#(form._((lib._syntax%2Fparse..rkt)._syntax-parse))" class="RktStxLink" pltdoc="x">syntax-parse</a></span> requires that every literal
refer to some binding. (I’ll sometimes refer to this requirement as
the <span style="font-style: italic">is-bound</span> property for short.) This requirement is
problematic in a different way. Specifically, this property cannot be
checked statically (that is, when the syntax-parse expression
containing the literal is compiled).</p><p>That might strike you as bizarre or unlikely. After all, you can
easily imagine checking that a <span class="RktSym"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket%2Fprivate%2Fstxcase-scheme..rkt)._syntax-rules))" class="RktStxLink" pltdoc="x">syntax-rules</a></span> macro, say,
satisfies the <span style="font-style: italic">is-bound</span> property. But in Racket, not every macro
uses <span class="RktSym"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket%2Fprivate%2Fstxcase-scheme..rkt)._syntax-rules))" class="RktStxLink" pltdoc="x">syntax-rules</a></span>, and—more importantly—not every bit of
syntax-analyzing code is a macro. And both of these facts have to do
with <a href="http://docs.racket-lang.org/reference/eval-model.html#(tech._phase)" class="techoutside" pltdoc="x"><span class="techinside">phases</span></a>.</p>
<a href="http://macrologist.blogspot.com/2011/09/syntax-parse-and-literals.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-30983799470574224222011-09-07T23:08:00.002-06:002011-09-28T03:30:14.473-06:00macros and literals<p>Macros often have associated auxiliary identifiers (sometimes called
<span style="font-style: italic">keywords</span> or <span style="font-style: italic">reserved words</span>, although both terms are
problematic in Racket). For example, <span class="RktSym"><a href="http://docs.racket-lang.org/reference/if.html#(form._((lib._racket%2Fprivate%2Fletstx-scheme..rkt)._cond))" class="RktStxLink" pltdoc="x">cond</a></span> has <span class="RktSym"><a href="http://docs.racket-lang.org/reference/if.html#(form._((lib._racket%2Fprivate%2Fletstx-scheme..rkt)._else))" class="RktStxLink" pltdoc="x">else</a></span>;
<span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._class))" class="RktStxLink" pltdoc="x">class</a></span> has <span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._public))" class="RktStxLink" pltdoc="x">public</a></span>, <span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._private))" class="RktStxLink" pltdoc="x">private</a></span>, etc;
<span class="RktSym"><a href="http://docs.racket-lang.org/reference/creatingunits.html#(form._((lib._mzlib%2Funit..rkt)._unit))" class="RktStxLink" pltdoc="x">unit</a></span> has <span class="RktSym"><a href="http://docs.racket-lang.org/reference/creatingunits.html#(form._((lib._mzlib%2Funit..rkt)._import))" class="RktStxLink" pltdoc="x">import</a></span> and <span class="RktSym"><a href="http://docs.racket-lang.org/reference/creatingunits.html#(form._((lib._mzlib%2Funit..rkt)._export))" class="RktStxLink" pltdoc="x">export</a></span>.</p>
<p>The fundamental question is what constitutes a use of an auxiliary
identifier, and there are two reasonable answers: <span style="font-style: italic">symbolic
equality</span> and <span style="font-style: italic">referential equality</span>. By symbolic equality I
mean, for example, that any identifier written with exactly the
letters <span class="stt">else</span> is accepted as an <span class="RktSym"><a href="http://docs.racket-lang.org/reference/if.html#(form._((lib._racket%2Fprivate%2Fletstx-scheme..rkt)._else))" class="RktStxLink" pltdoc="x">else</a></span> auxiliary form. By
referential equality I mean any identifier that <span style="font-style: italic">refers to</span>
(using the standard notions of binding, environments, etc) the binding
identified as the <span class="RktSym"><a href="http://docs.racket-lang.org/reference/if.html#(form._((lib._racket%2Fprivate%2Fletstx-scheme..rkt)._else))" class="RktStxLink" pltdoc="x">else</a></span> binding.</p>
<a href="http://macrologist.blogspot.com/2011/09/macros-and-literals.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-66270863313931057702011-08-31T20:38:00.001-06:002011-08-31T20:39:23.199-06:00asynchonous execution for databases, using places<p>
I added asynchronous execution to <a href="http://pre.racket-lang.org/docs/html/db/">my database library</a> yesterday using Racket's <a href="http://docs.racket-lang.org/reference/places.html">places</a>. The coding part took about an afternoon and part of an evening. The new code is a bit less than 300 lines, most of which is boring serialization and deserialization code, some of which will go away soon.
</p>
<p>
My database library contains two wire-protocol connection implementations (for PostgreSQL and MySQL) and two FFI-based connection implementations (for SQLite and ODBC). The wire-protocol implementations are more work, but they just use I/O ports, and Racket handles I/O pretty well. On the other hand, the entire Racket VM stops during an FFI call, because Racket threads are green threads.
</p>
<p>
Having all threads stop execution for FFI calls isn't much of a problem if the FFI calls are all short. If the FFI call is "execute this SQL statement," on the other hand, that can cause serious problems with responsiveness. (Of course, it still depends on how long the SQL statement in question takes to execute.)
</p>
<p>
ODBC provides the ability to execute some operations asynchronously—in theory. In practice, of all the drivers I had available on my development machines, only the DB2 driver actually supported asynchronous execution. Furthermore, the way one performs an asynchronous call—repeatedly calling a function with identical arguments until it returns something different—plays poorly with GC'd languages, where keeping memory locations identical from call to call requires more effort than it does in, say, C. In short, ODBC's asynchronous execution doesn't solve the interactivity problem.
</p>
<p>
Racket actually has multiple kinds of concurrency. In addition to (green) threads, Racket also has "futures" (true concurrency if it's not too much trouble, everything shared) and "places" (true concurrency for sure, almost nothing shared, message passing). You can't send higher-order data (functions, objects, etc) between places (rather, you would have to be clever about it), but database connections traffic in mostly first-order data structures, so it's relatively easy to create a connection proxy that dispatches to a real database connection running in a difference place.
</p>
<p>
The one exception, the single kind of higher-order data used by connections, is the prepared statement object. But it's possible to proxy those using a hash table and finalizers. (Actually, prepared statements use finalizers already to clean up resources, and the clean-up code is in the connection class, so I didn't even need to create a new prepared statement class.)
</p>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com0tag:blogger.com,1999:blog-21789874.post-1144533312248907432006-04-08T15:54:00.000-06:002011-09-28T03:14:39.372-06:00macros, parameters; binding, and reference<p>Danny Yoo had an interesting question on the plt-scheme mailing list
recently. At first it seemed like your standard non-hygienic, “I want
this to mean something in here” macro question. I used to group
macros into three levels of “hygienicness”: the hygienic ones, the
ones that are morally hygienic in that the names they introduce are
based on their input, and the totally non-hygienic ones that have a
fixed name that they stick into the program. An example of the middle
set is <span class="RktSym"><a href="http://docs.racket-lang.org/reference/define-struct.html#(form._((lib._racket%2Fprivate%2Fbase..rkt)._define-struct))" class="RktStxLink" pltdoc="x">define-struct</a></span>, and an example of the third set is a
loop construct that binds the name <span class="RktKw">yield</span> in its body.</p><p>The third class used to offend me from a purist’s (semi-purist?)
perspective. But it’s a very reasonable thing to want to do. Consider
the <span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._class))" class="RktStxLink" pltdoc="x">class</a></span> macro and the names it uses to do interesting
things: <span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._super))" class="RktStxLink" pltdoc="x">super</a></span>, <span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._public))" class="RktStxLink" pltdoc="x">public</a></span>, <span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._field))" class="RktStxLink" pltdoc="x">field</a></span>,
<span class="RktSym"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket%2Fprivate%2Fclass-internal..rkt)._init))" class="RktStxLink" pltdoc="x">init</a></span>, and so on. It depends on those particular names.</p><p>Danny Yoo was writing a generator library. He had a
<span class="RktKw">define-generator</span> form, used like this:</p><blockquote class="SCodeFlow"><p><span class="RktPn">(</span><span class="RktKw">define-generator</span><span class="hspace"> </span><span class="RktPn">(</span><span class="RktVar">name</span><span class="hspace"> </span><span class="RktPn">. </span><span class="RktVar">args</span><span class="RktPn">)</span><span class="hspace"> </span><span class="RktPn">. </span><span class="RktVar">body</span><span class="RktPn">)</span></p></blockquote><p>and he wanted <span class="RktKw">yield</span> to have a particular meaning inside of
the generator body. He had used the usual non-hygienic technique of
creating the right <span class="RktKw">yield</span> identifier using
<span class="RktSym"><a href="http://docs.racket-lang.org/reference/stxops.html#(def._((quote._~23~25kernel)._datum-~3esyntax))" class="RktValLink" pltdoc="x">datum->syntax</a></span>, but he was asking for other ideas.</p><p>Dave commented that non-hygienic macros typically do not play nicely
together; that is, it can be hard to write other macros that expand
into them, because you have to think about what version of the code
you want to bind the variable in, and you can’t always tell... it’s a
mess. Dave recommended creating two versions of the macro: a
non-hygienic front-end that forged the <span class="RktKw">yield</span> identifier, and
a hygienic back-end that did the actual implementation. People who
wanted to further abstract over generator definitions could use the
hygienic version.</p><p>But there’s another way to look at it, and that’s what I replied to
the mailing list. It got me thinking about the similarities between
macros and normal programming, and the different techniques we use.</p>
<a href="http://macrologist.blogspot.com/2006/04/macros-parameters-binding-and.html#more">Read more »</a>Ryan Culpepperhttp://www.blogger.com/profile/04275692281825651783noreply@blogger.com1