## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

John Launchbury Trevor Elliott

Motivating Example

Helper

Server

Client1 Xen hypervisor

Client2

Tester

• Tester talks with each of the VMs concurrently • Many possible behaviors are “correct” / “incorrect” • Timeouts, VMs dying, etc. • Subtle concurrency bugs in test framework

Concurrent Scripting

• Management of multiple resources

– Multiple paths – Timeouts and defaults – Processing of data and results

**• “Scripting” rather than “programming”
**

– Minimal concurrency paraphernalia

• E.g. thread identifiers, locks, etc. etc.

– Easy to get right

Orc Intuitions

l

O iftI

lift Lis t

IO

putMVar

Orc

<|>

List

readTVar

MonadIO

MonadPlus

Basic Examples

fplang :: Orc String fplang = return “Haskell” <|> return “ML” <|> return “Scheme”

fplang

“Haskell”

“ML”

“Scheme”

Basic Examples

metronome :: Orc () metronome = return () <|> (delay 2.5 >> metronome)

metronome

delay 2.5 ()

Basic Examples

metro :: Float -> Orc () metro j = return () <|> (delay j >> metro j) rsiReminder :: Orc () rsiReminder = do metro 3600 email “john at galois.com” “Get up and stretch!!”

Search

queens = fmap show (extend []) <|> return ("Computing 8-queens...") extend :: [Int] -> Orc [Int] extend xs = if length xs == 8 then return xs else do j <- liftList [1..8] guard $ not (conflict xs j) extend (j:xs) conflict :: [Int] -> Int conflict = ...

Concurrency point

MonadPlus operation

Search

*Main> printOrc (queens) Ans = "Computing 8-queens..." Ans = "[5,7,1,3,8,6,4,2]" Ans = "[6,4,2,8,5,7,1,3]" Ans = "[5,3,8,4,7,1,6,2]" Ans = "[4,2,7,3,6,8,5,1]" : *Main> printOrc (queens) Ans = "Computing 8-queens..." Ans = "[4,2,7,3,6,8,5,1]" Ans = "[6,4,7,1,8,2,5,3]" Ans = "[3,6,4,2,8,5,7,1]" Ans = "[2,7,3,6,8,5,1,4]" :

Orc API

newtype return (>>=) stop (<|>) eagerly (<+>) liftIO runOrc Orc a :: a -> Orc a :: Orc a -> (a -> Orc b) -> Orc b :: Orc a :: Orc a -> Orc a -> Orc a :: Orc a -> Orc (Orc a) :: Orc a -> Orc a -> Orc a :: IO a -> Orc a :: Orc a -> IO () Classes: Functor, Monad, Applicative, Alternative, MonadPlus, MonadIO

Using Eagerly

sync :: (a->b->c) -> Orc a -> Orc b -> Orc c sync f p q = do po <- eagerly p qo <- eagerly q

Monadic function application

• Entering the handle waits for the result • Synchronization

return f <*> po <*> qo notBefore:: Orc a -> Float -> Orc a p `notBefore` w = sync const p (delay w)

cut:: Orc a -> Orc a cut p = do po <- eagerly p po

cut:: Orc a -> Orc a cut = join . eagerly

Web Query

quotes :: Query -> Query -> Orc Quote quotes srcA srcB = do quoteA <- eagerly $ getQuote srcA quoteB <- eagerly $ getQuote srcB cut ( (return least <*> quoteA <*> quoteB) <|> (quoteA >>= threshold) <|> (quoteB >>= threshold) <|> (delay 25 >> (quoteA <|> quoteB)) <|> (delay 30 >> return noQuote))

A B

Need to book a ticket, under $300 if possible…

least x y = if price x < price y then x else y threshold x = guard (price x < 300) >> return x

quote

Counting the number of answers

Using <+>

accum

count :: Orc a -> Orc (Either a Int) count p = do accum <- newTVar 0 do x <- p modifyTVar accum (+1) return $ Left x <+> fmap Right (readTVar accum) newTVar readTVar

p

readTVar

:: MonadIO io => a -> io (TVar a) :: MonadIO io => TVar a -> io a

modifyTVar :: MonadIO io => TVar a -> (a -> a) -> io (a,a)

Control.Concurrent.STM.MonadIO

Orc Laws

Left-Return: Right-Return:

(return x >>= k) = k x (p >>= return) = p

**Bind-Associativity: ((p >>= k) >>= h) = (p >>= (k >=> h)) Stop-Identity: Par-Associativity: Left-Zero: Par-Bind:
**

p <|> stop = p

Par-Commutativity: p <|> q = q <|> p

p <|> (q <|> r) = (p <|> q) <|> r

(stop >>= k) = stop ((p <|> q) >>= k) = ((p >>= k) <|> (q >>= k))

Non-Laws

Bind-Par?:

p >>= (\x -> h x <|> k x) = (p >>= h) <|> (p >>= k)

Right-Zero?: p >> stop = stop

p `onlyUntil` done = cut (silent p <|> done) silent p = p >> stop

Repetition

Re-invoke the Orc argument after each success

repeating :: Orc a -> Orc a repeating p = do x <- p return x <|> repeating p

p

x

p

The computation bifurcates if p returns multiple values

x’

Permitting

An Orc term controls the execution of another

p q

(<#>) :: Orc a -> Orc b -> Orc a p <#> q = do end <- newEmptyMVar (p <+> silent (putMVar end ())) <|> silent (q `onlyUntil` takeMVar end)

q is run for its effects q is killed when p finishes

Controlling file downloads

Worked Example

downloads :: [(URL,Path)] -> IO () downloads fileList = do let downloads = map getFile fileList runOrc $ msum downloads getFile :: (URL,Path) -> Orc () getFile (url,path) = liftIO $ do res <- openURIString url case res of Left err -> putStrLn (url++": "++err) Right content -> writeFile ("Files/"++path) content

Runs all the downloads concurrently

IO code lifted into Orc

Bounded Parallelism

Unbounded parallelism

msum :: [Orc a] -> Orc a msum ps = foldr (<|>) stop ps

Add a bound

msum’ :: Orc Int[Orc[Orc-> Orc Orc a Int -> -> a] a] -> a msum’ bound ps = ...

Using Orc for the bound allows it to be dynamic

msum’ :: Orc Int -> [Orc a] -> Orc a msum’ bound ps = do t <- newTVar 0 msum (map (wrap t) ps) <#> setBound t bound wrap :: TVar Int -> Orc a -> Orc a wrap t p = do checkModifyTVar t (>0) (\x->x-1) p <+> silent (modifyTVar_ t (+1)) setBound :: TVar Int -> Orc Int -> Orc () setBound t bound = do n <- bound modifyTVar_ t (+n)

Bounded Parallelism

Launch p only if a resource is available

Update resource bound for each new bound

**Example: file downloads
**

downloads :: [(URL,Path)] -> IO () downloads fileList = do let downloads = map getFile fileList runOrc $ msum’ bounds downloads bounds = return 4

Runs all the downloads concurrently (bounded)

**Example: file downloads
**

downloads :: [(URL,Path)] -> IO () downloads fileList = do let downloads = map getFile fileList runOrc $ msum’ bounds downloads bounds = return 4 <|> promptUser promptUser :: Orc Int promptUser = do xs <- repeating $ liftIO (putStrLn "Enter bounds changes (+/-) " >> getLine) return (read xs)

Runs all the downloads concurrently (bounded)

Dynamic bound updating

Layered Implementation

• Layered implementation — layered semantics – Properties at one level depend on properties at the level below

multiple results thread control external effects

Orc Scripts Orc Monad HIO Monad IO Monad Transition Semantics

• What properties should Orc terms satisfy? – Hence, what properties should be built into HIO? • Unresolved question: what laws should the basic operations of the IO monad satisfy?

type Orc a = (a -> HIO ()) -> HIO () return x = \k -> k x p >>= h stop p <|> q = \k -> p (\x -> h x k) = \k -> return () = \k -> fork (p k) >> q k

Key Definitions

Discard the continuation

Duplicate the continuation

runOrc p = p (\x -> return ()) liftIO m = \k -> (liftIO m >>= k)

printOrc = runOrc $ do a <- p liftIO $ putStrLn ("Ans = " ++ show a)

HIO Monad

~~~~ ~ ~~ ~~~~ ~~~ ~~ ~~~~ ~~~~ ~~~ ~~ ~~ ~ ~~ ~~~~

• Don’t want the programmer to have to do explicit thread management – Nested groups of threads • Want richer equational theory than IO – e.g. by managing asynchronous exceptions

:: Group -> IO a}

~~~~ ~~~ ~~ ~~~ ~ ~~ ~~~~ a =

newtype HIO

HIO

~~~~ ~ = ~~

~~~~~ ~~ ~~ ~~~~~ ~~~ {inGroup ~~~~

type Group = (TVar Int, TVar Inhabitants) data Inhabitants Closed | Open [Entry]

data Entry = Thread ThreadId | Group Group

newPrimGroup :: IO Group register :: Entry -> Group -> IO ()

killGroup :: Group -> IO () increment, decrement, isZero :: Group -> IO ()

Concluding

• Orc in Haskell

– Ready for use today for concurrent scripting

• cabal install orc

**– Flexibility of embedded DSLs: discover new idioms – Design principles emerge
**

• Two kinds of communications • Many kinds of state elements

**• Ongoing exploration in concurrency
**

– What are the right abstractions?

**• Misra and Cook
**

– Thanks for their pioneering work on the Orc calculus

Backup

Eagerly Laws

Eagerly-Par: eagerly p >>= (\x -> k x <|> h) = (eagerly p >>= k) <|> h Eagerly-Swap:

do y <- eagerly p x <- eagerly q return (x,y) = do x <- eagerly q y <- eagerly p return (x,y)

Eagerly-IO: eagerly (liftIO m) >> p = (liftIO m >> stop) <|> p

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar forkM (p (\x -> putMVar r x)) k (liftIO $ readMVarM r)

Eagerly

read r ?

eagerly p p

eagerly p

p =

a

k

put r

k

• Give p a continuation that will store its result • Return the “value” that accesses that result for the then current continuation

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar forkM (p `saveOnce` r)) k (liftIO $ readMVar r)

Eagerly

saveOnce :: Orc a -> MVar a -> HIO () p `saveOnce` r = do p (\x -> putMVar r x)

• Give p a continuation that will store its result (but once only even if duplicated) • Return the “value” that accesses that result for the then current continuation

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar forkM (p `saveOnce` r)) k (liftIO $ readMVar r)

Eagerly

saveOnce :: Orc a -> MVar a -> HIO () p `saveOnce` r = do ticket <- newMVar () p (\x -> takeMVar ticket >> putMVar r x)

• Give p a continuation that will store its result (but once only even if duplicated) • Return the “value” that accesses that result for the then current continuation

eagerly :: Orc a -> Orc (Orc a) eagerly p = \k -> do r <- newEmptyMVar g <- newGroup local e $ forkM (p `saveOnce` (r,g)) k (liftIO $ readMVar r) saveOnce :: Orc a -> (MVar a,Group) -> HIO () p `saveOnce` (r,g) = do ticket <- newMVar () p (\x -> takeMVar ticket >> putMVar r x >> close g)

Eagerly

• Give p a continuation that will store its result (but once only even if duplicated) • Return the “value” that accesses that result for the then current continuation • Thread management can be carried over too

val val p

:: Orc a -> Orc a = \k -> do r <- newEmptyMVar g <- newGroup local e $ forkM (p `saveOnce` (r,g)) k (unsafePerformIO $ readMVar r)

Val

val p p

a

saveOnce :: Orc a -> (MVar a,Group) -> HIO () p `saveOnce` (r,g) = do ticket <- newMVar () p (\x -> takeMVar ticket >> putMVar r x >> close g)

• The implementation of val (the alternative that uses lazy thunks) is almost identical

quotesVal :: Query -> Query -> Orc Quote quotesVal srcA srcB = do quoteA <- val $ getQuote srcA quoteB <- val $ getQuote srcB cut ( publish (least quoteA quoteB) <|> (threshold quoteA) <|> (threshold quoteB)

Web Query

<|> (delay 25 >> (publish quoteA <|> publish quoteB)) <|> (delay 30 >> return noQuote)) publish :: NFData a => a -> Orc a publish x = deepseq x $ return x

• Good: use the lazy values directly • Bad: have to be careful about evaluation

- Tfpie2014 Submission 5uploaded bysnoobab
- Multicore Haskell Now!uploaded byDon Stewart
- A Wander Through GHC's New IO Libraryuploaded byDon Stewart
- An Introduction to Communicating Haskell Processesuploaded byDon Stewart
- Domain Specific Languages for Domain Specific Problemsuploaded byDon Stewart
- Improving Data Structuresuploaded byDon Stewart
- Engineering Large Projects in Haskell: A Decade of FP at Galoisuploaded byDon Stewart
- Typeclassopedia - HaskellWikiuploaded bynovorod
- The Design and Implementation of xmonaduploaded byDon Stewart
- Engineering Large Projects in a Functional Languageuploaded byDon Stewart
- Hackage, Cabal and the Haskell Platform: The Second Yearuploaded byDon Stewart
- Supercompilation for Haskelluploaded byDon Stewart
- Practical Haskell Programming: scripting with typesuploaded byDon Stewart
- Building a business with Haskell: Case Studies: Cryptol, HaLVM and Copilotuploaded byDon Stewart
- Modern Benchmarking in Haskelluploaded byDon Stewart
- 86992345 EnVision Users Guideuploaded byenisb
- NetworkSimTestCases-2015uploaded byThinh Tran Van
- A COMPARISON STUDY FOR INTRUSION DATABASE.pdfuploaded byAnonymous N22tyB6UN
- scanuploaded byanon-519708
- Customization Requirements NGHA LEO 03 May V4uploaded bymanish8446879368
- Knowledge Transition Planuploaded bylone0726
- Spares Enterprise Updated Processuploaded bysyedhasan30
- BPOuploaded byPrabhu Raja
- About-CASTuploaded byMuhd Daniel Asyraaf
- Book 1uploaded byrvrgopal2000
- How to Open the Client in SAP Systemuploaded byOmer Farooq
- LOC Configurationuploaded bysuprasannap
- DEV1213 - Sublist Line Approval Without Scripting-FINAL_1523308484324001ZQM1uploaded byRamteja Goud Kosuri
- New Resumeuploaded byRavuri Veeraiah Chowdary
- Transit tuploaded by1lvlup

- The Semantics of Asynchronous Exceptionsuploaded byDon Stewart
- Multicore programming in Haskelluploaded byDon Stewart
- Loop Fusion in Haskelluploaded byDon Stewart
- Engineering Large Projects in a Functional Languageuploaded byDon Stewart
- Hackage, Cabal and the Haskell Platform: The Second Yearuploaded byDon Stewart
- Multicore Haskell Now!uploaded byDon Stewart
- Practical Haskell Programming: scripting with typesuploaded byDon Stewart
- Evaluation strategies and synchronization: things to watch foruploaded byDon Stewart
- Building a business with Haskell: Case Studies: Cryptol, HaLVM and Copilotuploaded byDon Stewart
- Galois Tech Talk: A Scalable Io Manager for GHCuploaded byDon Stewart
- Modern Benchmarking in Haskelluploaded byDon Stewart
- Specialising Generators for High-Performance Monte-Carlo Simulation ... in Haskelluploaded byDon Stewart
- Improving Data Structuresuploaded byDon Stewart
- Stream Fusion for Haskell Arraysuploaded byDon Stewart
- Engineering Large Projects in Haskell: A Decade of FP at Galoisuploaded byDon Stewart
- Haskell: Batteries Includeduploaded byDon Stewart
- The Design and Implementation of xmonaduploaded byDon Stewart
- The Birth of the Industrial Haskell Group : CUFP 2009uploaded byDon Stewart
- The Haskell Platform: Status Reportuploaded byDon Stewart
- Haskell Arrays Accelerated with GPUsuploaded byDon Stewart
- Multicore Programming in Haskell Now!uploaded byDon Stewart
- Supercompilation for Haskelluploaded byDon Stewart

Close Dialog## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Loading