I will humbly suggest Seesaw.
Here's a REPL-based tutorial that assumes no Java or Swing knowledge.
Seesaw's a lot like what @tomjen suggests. Here's "Hello, World":
(use 'seesaw.core)
(-> (frame :title "Hello"
:content "Hello, Seesaw"
:on-close :exit)
pack!
show!)
and here's @Abhijith and @dsm's example, translated pretty literally:
(ns seesaw-test.core
(:use seesaw.core))
(defn handler
[event]
(alert event
(str "<html>Hello from <b>Clojure</b>. Button "
(.getActionCommand event) " clicked.")))
(-> (frame :title "Hello Swing" :on-close :exit
:content (button :text "Click Me" :listen [:action handler]))
pack!
show!)
