This lesson is in the early stages of development (Alpha version)

Programming with Julia

Introduction

Overview

Teaching: 5 min
Exercises: 0 min
Questions
  • What is Julia?

  • Why use Julia?

Objectives
  • Explain the difference between interpreted and compiled programming languages

  • Compare how composing works in Julia and some common programming languages

What is a programming language?

A programming language mediates between the natural language of humans and the machine instructions of a computer. The human specifies what the computer should compute on a high level using the programming language. This specification will be translated to machine instructions, the so called assembly code, which will be executed by the processor (CPU, GPU, …).

Interpreting and compiling

This translation happens differently depending on the programming language you use. There are mainly two different techniques: compiling and interpreting. Interpreted languages such as Python and R translate instructions one at a time, while compiled languages like C and Fortran take whole documents, analyze the structure of the code, and perform optimizations before translating it to machine code.

This leads to more efficient machine instructions of the compiled code at the cost of less flexibility and more verbose code. Most prominently, compiled languages need an explicit type declaration for each variable.

Why Julia?

Julia is a programming language that superficially looks like an interpreted language and mostly behaves like one. But before each function is executed it will be compiled just in time.

Thus you get the flexibility of an interpreted language and the execution speed of a compiled language at the cost of waiting a bit longer for the first execution of any function.

There is another aspect of Julia that makes it interesting and that is the way packages compose. This is captured the best by an analogy from Sam Urmy:

Say you want a toy truck.

The Python/R solution is to look for the appropriate package–like buying a Playmobil truck. It comes pre-manufactured, well-engineered and tested, and does 95% of what you would ever want a toy truck to do.

The Fortran/C solution is to build the truck from scratch. This allows total customization and you can optimize the features and performance however you want. The downside is that it takes more time, you need woodworking skills, and might hurt yourself with the power tools.

The Julia solution is like Legos. You can get the truck kit if you want–which will require a bit more assembly than the Playmobil, but way less than building it from scratch. Or, you can get the component pieces and assemble the truck to your own specs. There’s no limit to what you can put together, and because the pieces all have the same system of bumps, everything snaps together quickly and easily.

Trucks

OK, sure. Toy trucks are like linear algebra, though, a common request, and every “toy system” will have an implementation that works basically fine. But what if you want a time-traveling sailing/space ship with lasers AND dragon wings? And it should be as easy to build and use as a basic dump truck?

Vessel

There’s a reason that only Lego ever made anything like Dr. Cyber’s Flying Time Vessel!

Originally posted on Discourse.

Key Points

  • Julia is a just-in-time compiled language

  • Julia packages compose well


Using the REPL

Overview

Teaching: 18 min
Exercises: 2 min
Questions
  • How to use the REPL?

Objectives
  • Explore basic functionality of input.

  • Learn how to declare variables.

  • Learn about REPL modes.

Entering the REPL

Melissa and her classmates open a terminal and launch julia:

julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0 (2021-11-30)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia>

This is the so-called REPL, which stands for read-evaluate-print-loop. The interactive command-line REPL allows quick and easy execution of Julia statements.

Like the terminal, the Julia REPL has a prompt, where it awaits input:

julia>

Implicit prompt

Most of the code boxes that follow do not show the julia> prompt, even though it’s there in the REPL. Why?

It’s important to delineate input (what you type) and output (how the machine responds). The prompt can be confusing, so it is excluded. You may assume that any Julia box prepends the prompt on each line of input.

Variables

The first thing they try is to perform basic arithmetic operations:

1 + 4 * 7.3
30.2

That works as expected. It is also possible to bind a name to a value via the assignment operator =, which makes it easier to refer to the value later on. These names are called variables.

distance = 30.2
distance_x_2 = 2 * distance
60.4

Melissa notices that assignment also returns the value. She can also check which variables are defined in the current session by running

varinfo()
  name                    size summary
  –––––––––––––––– ––––––––––– –––––––
  Base                         Module
  Core                         Module
  InteractiveUtils 270.164 KiB Module
  Main                         Module
  ans                  8 bytes Float64
  distance             8 bytes Float64
  distance_x_2         8 bytes Float64

Unicode

In Julia, Unicode characters are also allowed as variables like α = 2. Unicode characters can be entered by a backslash followed by their LaTeX name and then pressing tab (in this case \alphatab).

REPL-modes

Unfortunately Melissa can’t remember the LaTeX name of ∂ so she copies the character, presses ? for help mode,

?

pastes the ∂ character, then presses enter:

help?> 
"∂" can be typed by \partial<tab>

Great! This way she can easily look up the names she needs. She gets back to normal mode by pressing backspace.

Another useful mode is the shell mode that can be entered by pressing ;. The prompt has now changed:

;
shell>

Shell mode can be used to issue commands to the underlying shell, but don’t confuse it with an actual shell: special shell syntax like piping won’t work. Like before, hit backspace to get back to the Julia prompt.

Hello, shell>!

Use the shell mode to create a file called hello.jl with the nano terminal text editor. Inside that file write the simple hello world program print("Hello World").

Check the content of the file using cat hello.jl and then run the program using julia hello.jl.

Solution

;
shell> nano hello.jl
shell> cat hello.jl
print("Hello World")
shell> julia hello.jl
Hello World

backspace

Finally there is package mode that is entered with ] which is used for package management, which will be covered later on:

]
pkg>

To exit shell, help or pkg mode, hit backspace.

Key Points

  • The REPL reads the given input, evaluates the given expression and prints the resulting output to the user.

  • Pressing ? enters help mode.

  • Pressing ; enters shell mode.

  • Pressing ] enters pkg mode.


Julia type system

Overview

Teaching: 15 min
Exercises: 5 min
Questions
  • What is the use of types?

  • How are types organized in Julia?

Objectives
  • Understand the structure of the type tree.

  • Know how to traverse the type tree.

  • Know how to build mutable and immutable types.

Structuring variables

Melissa wants to keep the variables corresponding to the trebuchet (counterweight, release_angle) separate from the variables coming from the environment (wind, target_distance). That is why she chooses to group them together using structures. There are two structure types:

Since Melissa wants to change the parameters of the trebuchet, she uses a mutable struct for it. But she cannot influence the environment and thus uses a struct for those values.

mutable struct Trebuchet
  counterweight::Float64
  release_angle::Float64
end

struct Environment
  wind::Float64
  target_distance::Float64
end

Types and hierarchy

Here ::Float64 is a type specification, indicating that this variable should be a 64-bit floating point number, and :: is an operator that is read “is an instance of.” If Melissa hadn’t specified the type, the variables would have the type Any by default.

In Julia every type can have only one supertype, so let’s count how many types are between Float64 and Any:

  1. supertype(Float64)
    
    AbstractFloat
    
  2. supertype(AbstractFloat)
    
    Real
    
  3. supertype(Real)
    
    Number
    
  4. supertype(Number)
    
    Any
    

So we have the relationship Float64 <: AbstractFloat <: Real <: Number <: Any where <: is the subtype operator, used here to mean the item on the left “is a subtype of” the item on the right.

Float64 is a concrete type, which means that you can actually create objects of this type. For example 1.0 is an object of type Float64. We can check this at the REPL using either (or both) the typeof function or the isa operator:

typeof(1.0)
1.0 isa Float64
Float64
true

All the other types are abstract types that are used to address groups of types. For example, if we declare a variable as a::Real then it can be bound to any value that is a subtype of Real.

Let’s quickly check what are all the subtypes of Real:

subtypes(Real)
4-element Array{Any,1}:
 AbstractFloat
 AbstractIrrational
 Integer
 Rational

This way the types form a tree with abstract types on the nodes and concrete types as leaves. Have a look at this visualization of all subtypes of Number: Type_tree-Number

Is it Real?

For which of the following types T would the following return false?

1.0 isa T
  1. Real
  2. Number
  3. Float64
  4. Integer

Solution

The correct answer is 4: while 1 is an integer, 1.0 is a floating-point value.

Creating a subtype

A concrete type can be made a subtype of an abstract type with the subtype operator <:. Since Trebuchet contains several fields that are mutable Melissa thinks it is a good idea to make it a subtype of AbstractVector.

Caveat: Redefining Structs

mutable struct Trebuchet <: AbstractVector{Float64}
  counterweight::Float64
  release_angle::Float64
end
ERROR: invalid redefinition of constant Trebuchet
Stacktrace:
 [1] top-level scope
   @ REPL[9]:1

This error message is clear: you’re not allowed to define a struct using a name that’s already in use.

Restart the REPL

In Julia it is not very easy to redefine structs. It is necessary to restart the REPL to define the new definition of Trebuchet, or take a different name instead.

Melissa decides to keep going and come back to this later.

Key Points

  • In Julia types have only one direct supertype.


Using the package manager

Overview

Teaching: 20 min
Exercises: 0 min
Questions
  • Where do I find packages?

  • How do I add packages?

  • How can I use packages?

Objectives
  • Learn to add packages using pkg-mode

  • Learn to resolve name conflicts

  • Learn to activate environments

The package manager

Now it is time for Melissa and their mates to simulate the launch of the trebuchet. The necessary equations are really complicated, but an investigation on JuliaHub revealed that someone already implemented these and published it as the Julia package Trebuchet.jl. That saves some real work.

Melissa enters package mode by pressing ]:

]

The julia> prompt becomes a blue pkg> prompt that shows the Julia version that Melissa is running.

After consulting the documentation she knows that the prompt is showing the currently activated environment and that this is the global environment that is activated by default.

However, she doesn’t want to clutter the global environment when working on her project. The default global environment is indicated with (@v1.x) before the pkg> prompt, where x is the minor version number of julia, so on julia 1.7 it will look like (@v1.7). To create a new environment she uses the activate function of the package manager:

(@v1.x) pkg> activate projects/trebuchet

In this environment she adds the Trebuchet package from its open source code repository on GitHub by typing

(trebuchet) pkg> add Trebuchet#master

Melissa quickly recognizes that far more packages are being installed than just Trebuchet. These are the dependencies of Trebuchet. From the output

[...]
Updating `[...]/projects/trebuchet/Project.toml`
  [98b73d46] + Trebuchet v0.2.1
  Updating `[...]/projects/trebuchet/Manifest.toml`
  [1520ce14] + AbstractTrees v0.3.3
  [79e6a3ab] + Adapt v1.1.0
  [...]

she sees that two files were created: Project.toml and Manifest.toml.

The project file Project.toml only contains the packages needed for her project, while the manifest file Manifest.toml records the direct and indirect dependencies as well as their current version, thus providing a fully reproducible record of the code that is actually executed. “That is really handy when I want to share my work with the others,” thinks Melissa.

After the installation finished she can check the packages present in her environment.

(trebuchet) pkg> status
      Status `projects/trebuchet/Project.toml`
  [98b73d46] Trebuchet v0.2.1 `https://github.com/FluxML/Trebuchet.jl#master`

Melissa can get back to the global environment using activate without any parameters.

Why use GitHub?

Melissa could have added the JuliaHub version of Trebuchet.jl by typing

(trebuchet) pkg> add Trebuchet

However, that “release” version of the code is missing some important features and, more important for learning, it has very little documentation. The “development” version, represented by the “master” branch on the GitHub repository, provides a function and its documentation that Melissa needs to use later on.

If you know a package is stable, go ahead and install the default version registered on JuliaHub. Otherwise, it’s good to check how different that version is from the current state of the software project. Click through the link under “Repository” on the JuliaHub package page.

Using and importing packages

Now that Melissa added the package to her environment, she needs to load it. Julia provides two keywords for loading packages: using and import.

The difference is that import brings only the name of the package into the namespace and then all functions in that package need the name in front (prefixed). But packages can define a list of function names to export, which means the functions should be brought into the user’s namespace when he loads the package with using. This makes working at the REPL more convenient.

Name conflicts

It may happen that name conflicts arise. For example Melissa defined a structure named Trebuchet, but the package she added to the environment is also named Trebuchet. Now she would get an error if she tried to import/using it directly. One solution is to assign a nickname or alias to the package upon import using the keyword as:

import Trebuchet as Trebuchets

Key Points

  • Find packages on JuliaHub

  • add packages using pkg> add

  • use many small environments rather than one big environment


Write functions!

Overview

Teaching: 15 min
Exercises: 5 min
Questions
  • How do I call a function?

  • Where can I find help about using a function?

  • What are methods?

Objectives
  • usage of positional and keyword arguments

  • defining named and anonymous functions

  • reading error messages

Working with functions

Now that Melissa successfully installed the package she wants to figure out what she can do with it.

Julia’s Base module offers a handy function for inspecting other modules called names. Let’s look at its docstring; remember that pressing ? opens the help?> prompt:

help?> names
    names(x::Module; all::Bool = false, imported::Bool = false)

    Get an array of the names exported by a Module, excluding deprecated names.
    If all is true, then the list also includes non-exported names defined in
    the module, deprecated names, and compiler-generated names. If imported is
    true, then names explicitly imported from other modules are also included.

    As a special case, all names defined in Main are considered "exported", 
    since it is not idiomatic to explicitly export names from Main.

In Julia we have two types of arguments: positional and keyword, separated by a semi-colon.

  1. Positional arguments are determined by their position and thus the order in which arguments are given to the function matters.
  2. Keyword arguments are passed as a combination of the keyword and the value to the function. They can be given in any order, but they need to have a default value.

Positional and keyword arguments

Let’s take a closer look at the signature of the names function:

names(x::Module; all::Bool = false, imported::Bool = false)

It takes three arguments:

  1. x, a positional argument of type Module,
    followed by a ;
  2. all, a keyword argument of type Bool with a default value of false
  3. imported, another Bool keyword argument that defaults to false

Suppose Melissa wanted to get all names of the Trebuchets module, including those that are not exported. What would the function call look like?

  1. names(Trebuchets, true)
  2. names(Trebuchets, all = true)
  3. names(Trebuchets; all = true)
  4. names(Trebuchets, all)
  5. Answer 2. and 3.

Solution

  1. Both arguments are present, but true is presented without a keyword. This throws a MethodError: no method matching names(::Module, ::Bool)
  2. This is a correct call.
  3. This is also correct: you can specify where the positional arguments end with the ;, but you do not have to.
  4. Two arguments are present, but the keyword all is not assigned a value. This throws a MethodError: no method matching names(::Module, ::typeof(all))
  5. This is the most correct answer.

Melissa goes ahead and executes

names(Trebuchets)
6-element Vector{Symbol}:
 :Trebuchet
 :TrebuchetState
 :run
 :shoot
 :simulate
 :visualise

which yields the exported names of the Trebuchets module. By convention types are named with CamelCase while functions typically have snake_case. Since Melissa is interested in simulating shots, she looks at the shoot function from Trebuchets (again, using ?):

help?> Trebuchets.shoot
  shoot(ws, angle, w)
  shoot((ws, angle, w))

  Shoots a Trebuchet with weight w in kg. Releases the weight at the release
  angle angle in radians. The current wind speed is ws in m/s. 
  Returns (t, dist), with travel time t in s and travelled distance dist in m.

Methods

Here we see that the shoot function has two different methods. The first one takes three arguments, while the second takes a Tuple with three elements.

Now she is ready to fire the first shot.

Trebuchets.shoot(5, 0.25pi, 500)
(TrebuchetState(Trebuchet.Lengths{Float64}(1.52, 2.07, 0.533, 0.607, 2.08, 0.831, 0.0379),
                Trebuchet.Masses{Float64}(226.0, 0.149, 4.83),
                Trebuchet.Angles{Float64}(-0.503, 1.32, 1.46),
                Trebuchet.AnglularVelocities{Float64}(-5.57, 7.72, -25.4),
                Trebuchet.Constants{Float64}(5.0, 1.0, 1.0, 9.81, 0.785),
                Trebuchet.Inertias{Float64}(0.042, 2.73),
                Val{:End}(), 
                60.0,
                Trebuchet.Vec(117.8, -1.524),
                Trebuchet.Vec(10.79, -21.45),
                Solution(394),
                0,
                Val{:Released}()
                ),
 117.8
)

That is a lot of output, but Melissa is actually only interested in the distance, which is the second element of the tuple that was returned. So she tries again and grabs the second element this time:

Trebuchets.shoot(5, 0.25pi, 500)[2]
117.8

which means the shot traveled approximately 118 m.

Defining functions

Melissa wants to make her future work easier and she fears she might forget to take the second element. That’s why she puts it together in a function like this:

function shoot_distance(windspeed, angle, weight)
    Trebuchets.shoot(windspeed, angle, weight)[2]
end

Implicit return

Note that Melissa didn’t have to use the return keyword, since in Julia the value of the last line will be returned by default. But she could have used an explicit return and the function would behave the same.

Now Melissa can just call her wrapper function:

shoot_distance(5, 0.25pi, 500)
117.8

Adding methods

Since Melissa wants to work with the structs Trebuchet and Environment, she adds another convenience method for those:

function shoot_distance(trebuchet::Trebuchet, env::Environment)
    shoot_distance(env.wind, trebuchet.release_angle, trebuchet.counterweight)
end

This method will call the former method and pass the correct fields from the Trebuchet and Environment structures.

Slurping and splatting

By peeking into the documentation, Melissa discovers that she doesn’t need to explicitly declare all the input arguments. Instead she can slurp the arguments in the function definition and splat them in the function body using three dots (...) like this:

function shoot_distance(args...) # slurping
    Trebuchets.shoot(args...)[2] # splatting
end

Anonymous functions

Sometimes it is useful to have a new function and not have to come up with a new name. These are anonymous functions. They can be defined with either the so-called stabby lambda notation,

(windspeed, angle, weight) -> Trebuchets.shoot(windspeed, angle, weight)[2]

or in long form, by omitting the name:

function (windspeed, angle, weight)
    Trebuchets.shoot(windspeed, angle, weight)[2]
end

Errors and macros

Melissa would like to set the fields of a Trebuchet using an index. She writes

trebuchet[1] = 2
ERROR: MethodError: no method matching setindex!(::Trebuchet, ::Int64, ::Int64)
Stacktrace:
 [1] top-level scope
   @ REPL[4]:1

The error tells her two things:

  1. a function named setindex! was called
  2. it didn’t have a method for Trebuchet

Melissa wants to add the missing method to setindex! but she doesn’t know where it is defined. There is a handy macro named @which that obtains the module where the function is defined.

Macros

Macro names begin with @ and they don’t need parentheses or commas to delimit their arguments. Macros can transform any valid Julia expression and are quite powerful. They can be expanded by prepending @macroexpand to the macro call of interest.

@which setindex!
Base

Now Melissa knows she needs to add a method to Base.setindex! with the signature (::Trebuchet, ::Int64, ::Int64).

Key Points

  • You can think of functions being a collection of methods

  • Keep the number of positional arguments low

  • Macros transform Julia expressions


Control flow

Overview

Teaching: 60 min
Exercises: 60 min
Questions
  • What are for and while loops?

  • How to use conditionals?

  • What is an interface?

Objectives

Conditionals

Now that Melissa knows which method to add she thinks about the implementation.

If the index is 1 she wants to set counterweight while if the index is 2 she wants to set release_angle and since these are the only two fields she wants to return an error if anything else comes in. In Julia the keywords to specify conditions are if, elseif and else, closed with an end. Thus she writes

function Base.setindex!(trebuchet::Trebuchet, v, i::Int)
    if i === 1
        trebuchet.counterweight = v
    elseif i === 2
        trebuchet.release_angle = v
    else
        error("Trebuchet only accepts indices 1 and 2, yours is $i")
    end
end

Interfaces

setindex! is actually one function of a widespread interface in the Julia language: AbstractArrays. An interface is a collection of methods that are all implemented by a certain type. For example, the Julia manual lists all methods that a subtype of AbstractArray need to implement to adhere to the AbstractArray interface:

If Melissa implements this interface for the Trebuchet type, it will work with every function in Base that accepts an AbstractArray.

She also needs to make Trebuchet a proper subtype of AbstractArray as she tried in the types episode. Therefore she restarts her REPL and redefines Trebuchet and Environment, as well as the slurp-and-splat shoot_distance function:

import Trebuchet as Trebuchets

mutable struct Trebuchet <: AbstractVector{Float64}
  counterweight::Float64
  release_angle::Float64
end

struct Environment
    wind::Float64
    target_distance::Float64
end

function shoot_distance(args...)
    Trebuchets.shoot(args...)[2]
end

Then she goes about implementing the AbstractArray interface.

Implement the AbstractArray interface for Trebuchet

Now we know enough to actually implement the AbstractArray interface. You don’t need to implement the optional methods.

Hint: Take a look at the docstrings of getfield and tuple.

Solution

Base.size(trebuchet::Trebuchet) = tuple(2)
Base.getindex(trebuchet::Trebuchet, i::Int) = getfield(trebuchet, i)
function Base.setindex!(trebuchet::Trebuchet, v, i::Int)
    if i === 1
        trebuchet.counterweight = v
    elseif i === 2
        trebuchet.release_angle = v
    else
        error("Trebuchet only accepts indices 1 and 2, yours is $i")
    end
end

With the new Trebuchet defined with a complete AbstractArray interface, Melissa tries again to modify a counterweight by index:

trebuchet = Trebuchet(500, 0.25pi)
2-element Trebuchet:
 500.0
   0.7853981633974483
trebuchet[1] = 2
2
trebuchet
2-element Trebuchet:
   2.0
   0.7853981633974483

Loops

Now Melissa knows how to shoot the virtual trebuchet and get the distance of the projectile, but in order to aim she needs to take a lot of trial shots in a row. She wants her trebuchet to only shoot a hundred meters.

She could execute the function several times on the REPL with different parameters, but that gets tiresome quickly. A better way to do this is to use loops.

But first Melissa needs a way to improve her parameters.

Digression: Gradients

The shoot_distance function takes three input parameters and returns one value (the distance). Whenever we change one of the input parameters, we will get a different distance.

The gradient of a function gives the direction in which the return value will change when each input value changes.

Since the shoot_distance function has three input parameters, the gradient of shoot_distance will return a 3-element Array: one direction for each input parameter.

Thanks to automatic differentiation and the Julia package ForwardDiff.jl gradients can be calculated easily.

Melissa uses the gradient function of ForwardDiff.jl to get the direction in which she needs to change the parameters to make the largest difference.

Do you remember?

What does Melissa need to write into the REPL to install the package ForwardDiff?

  1. ] install ForwardDiff
  2. add ForwardDiff
  3. ] add ForwardDiff.jl
  4. ] add ForwardDiff

Solution

The correct solution is 4: ] to enter pkg mode, then

pkg> add ForwardDiff
using ForwardDiff: gradient

imprecise_trebuchet = Trebuchet(500.0, 0.25pi);
environment = Environment(5.0, 100.0);

grad = gradient(x -> (shoot_distance([environment.wind, x[2], x[1]])
                      - environment.target_distance),
                imprecise_trebuchet)
2-element Vector{Float64}:
  -0.02210101414630771
 -47.191737880211264

Melissa now changes her arguments a little bit in the direction of the gradient and checks the new distance.

julia> better_trebuchet = imprecise_trebuchet - 0.05 * grad;

julia> shoot_distance([5, better_trebuchet[2], better_trebuchet[1]])
58.871526223121755

Great! That didn’t shoot past the target, but instead it landed a bit too short.

Experiment

How far can you change the parameters in the direction of the gradient, such that it still improves the distance?

Evaluation

Try a bunch of values!

  • better_trebuchet = imprecise_trebuchet - 0.04 * grad
    shoot_distance([environment.wind, better_trebuchet[2], better_trebuchet[1]])
    120.48753521261001
    
  • better_trebuchet = imprecise_trebuchet - 0.03 * grad
    shoot_distance([environment.wind, better_trebuchet[2], better_trebuchet[1]])
    107.80646596787481
    
  • better_trebuchet = imprecise_trebuchet - 0.02 * grad
    shoot_distance([environment.wind, better_trebuchet[2], better_trebuchet[1]])
    33.90699307740854
    
  • better_trebuchet = imprecise_trebuchet - 0.025 * grad
    shoot_distance([environment.wind, better_trebuchet[2], better_trebuchet[1]])
    75.87613276409223
    

Looks like the “best” trebuchet for a target 100 m away will be between 2.5% and 3% down the gradient from the imprecise trebuchet.

For loops

Now that Melissa knows it is going in the right direction she wants to automate the additional iterations. She writes a new function aim, that performs the application of the gradient N times.

function aim(trebuchet, environment; N = 10, η = 0.05)
           better_trebuchet = copy(trebuchet)
           for _ in 1:N
               grad = gradient(x -> (shoot_distance([environment.wind, x[2], x[1]]) 
                                     - environment.target_distance),
                               better_trebuchet)
               better_trebuchet -= η * grad
               # short form of `better_trebuchet = better_trebuchet - η * grad`
           end
           return Trebuchet(better_trebuchet[1], better_trebuchet[2])
       end

better_trebuchet  = aim(imprecise_trebuchet, environment);

shoot_distance(environment.wind, better_trebuchet[2], better_trebuchet[1])
90.14788588648652

Explore

Play around with different inputs of N and η. How close can you come?

Reason

This is a highly non-linear system and thus very sensitive. The distances across different values for the counterweight and the release angle α look like this:

distance-surface

Aborting programs

If a call takes too long, you can abort it with Ctrl-c

While loops

Melissa finds the output of the above aim function too unpredictable to be useful. That’s why she decides to change it a bit. This time she uses a while-loop to run the iterations until she is sufficiently near her target.

(Hint: ε is \epsilontab, and η is \etatab.)

function aim(trebuchet::Trebuchet, environment::Environment; ε = 0.1, η = 0.05)
    better_trebuchet = copy(trebuchet)
    hit = x -> (shoot_distance([environment.wind, x[2], x[1]])
                - environment.target_distance)
            while abs(hit(better_trebuchet)) > ε
                grad = gradient(hit, better_trebuchet)
                better_trebuchet -= η * grad
            end
            return Trebuchet(better_trebuchet[1], better_trebuchet[2])
        end

better_trebuchet = aim(imprecise_trebuchet, environment);

shoot_distance(better_trebuchet, environment)
100.0975848073789

That is more what she had in mind. Your trebuchet may be tuned differently, but it should hit just as close as hers.

Key Points

  • Interfaces are informal

  • Use for loops for a known number of iterations and while loops for an unknown number of iterations.

  • Julia packages compose nicely.


Using Modules

Overview

Teaching: 15 min
Exercises: 0 min
Questions
  • What’s the purpose of modules?

Objectives
  • Structure your code using modules

  • Use Revise.jl to track changes

Modules

Melissa now has a bunch of definitions in her running Julia session and using the REPL for interactive exploration is great, but it is more and more taxing to keep in mind what is defined, and all the definitions are lost once she closes the REPL.

That is why she decides to put her code in a file. She opens up her text editor and creates a file called aim_trebuchet.jl in the current working directory and pastes the code she got so far in there. This is what it looks like:

import Trebuchet as Trebuchets
using ForwardDiff: gradient

mutable struct Trebuchet <: AbstractVector{Float64}
  counterweight::Float64
  release_angle::Float64
end

Base.copy(trebuchet::Trebuchet) = Trebuchet(trebuchet.counterweight, trebuchet.release_angle)

Base.size(trebuchet::Trebuchet) = tuple(2)

Base.getindex(trebuchet::Trebuchet, i::Int) = getfield(trebuchet, i)

function Base.setindex!(trebuchet::Trebuchet, v, i::Int)
    if i === 1
        trebuchet.counterweight = v
    elseif i === 2
        trebuchet.release_angle = v
    else
        error("Trebuchet only accepts indices 1 and 2, yours is $i")
    end
end

struct Environment
  wind::Float64
  target_distance::Float64
end

function shoot_distance(windspeed, angle, weight)
    Trebuchets.shoot(windspeed, angle, weight)[2]
end

function shoot_distance(args...)
    Trebuchets.shoot(args...)[2]
end

function shoot_distance(trebuchet::Trebuchet, env::Environment)
    shoot_distance(env.wind, trebuchet.release_angle, trebuchet.counterweight)
end

function aim(trebuchet::Trebuchet, environment::Environment; ε = 1e-1, η = 0.05)
    better_trebuchet = copy(trebuchet)
    hit = x -> (shoot_distance([environment.wind, x[2], x[1]]) - environment.target_distance)
    while abs(hit(better_trebuchet)) > ε
        grad = gradient(hit, better_trebuchet)
        better_trebuchet -= η * grad
    end
    return Trebuchet(better_trebuchet[1], better_trebuchet[2])
end

imprecise_trebuchet = Trebuchet(500.0, 0.25pi)

environment = Environment(5, 100)

precise_trebuchet = aim(imprecise_trebuchet, environment)

shoot_distance(precise_trebuchet, environment)

Now Melissa can run include("aim_trebuchet.jl") in the REPL to execute her code.

She also recognizes that she has a bunch of definitions at the beginning that she doesn’t need to execute more than once in a session and some lines at the end that use these definitions which she might run more often. She will split these in two separate files and put the definitions into a module. The module will put the definitions into their own namespace which is the module name. This means Melissa would need to put the module name before each definition if she uses it outside of the module. But she remembers from the pkg episode that she can export names that don’t need to be prefixed.

She names her module MelissasModule and accordingly the file MelissasModule.jl. From this module she exports the names aim, shoot_distance, Trebuchet and Environment. This way she can leave her other code unchanged.

module MelissasModule
import Trebuchet as Trebuchets
using ForwardDiff: gradient

export aim, shoot_distance, Trebuchet, Environment

mutable struct Trebuchet <: AbstractVector{Float64}
  counterweight::Float64
  release_angle::Float64
end

Base.copy(trebuchet::Trebuchet) = Trebuchet(trebuchet.counterweight, trebuchet.release_angle)

Base.size(trebuchet::Trebuchet) = tuple(2)

Base.getindex(trebuchet::Trebuchet, i::Int) = getfield(trebuchet, i)

function Base.setindex!(trebuchet::Trebuchet, v, i::Int)
    if i === 1
        trebuchet.counterweight = v
    elseif i === 2
        trebuchet.release_angle = v
    else
        error("Trebuchet only accepts indices 1 and 2, yours is $i")
    end
end

struct Environment
  wind::Float64
  target_distance::Float64
end

function shoot_distance(windspeed, angle, weight)
    Trebuchets.shoot(windspeed, angle, weight)[2]
end

function shoot_distance(args...)
    Trebuchets.shoot(args...)[2]
end

function shoot_distance(trebuchet::Trebuchet, env::Environment)
    shoot_distance(env.wind, trebuchet.release_angle, trebuchet.counterweight)
end

function aim(trebuchet::Trebuchet, environment::Environment; ε = 1e-1, η = 0.05)
    better_trebuchet = copy(trebuchet)
    hit = x -> (shoot_distance([environment.wind, x[2], x[1]]) - environment.target_distance)
    while abs(hit(better_trebuchet)) > ε
        grad = gradient(hit, better_trebuchet)
        better_trebuchet -= η * grad
    end
    return Trebuchet(better_trebuchet[1], better_trebuchet[2])
end
end # MelissasModule

The rest of the code goes to a file she calls MelissasCode.jl.

using .MelissasModule

imprecise_trebuchet = Trebuchet(500.0, 0.25pi)
environment = Environment(5, 100)
precise_trebuchet = aim(imprecise_trebuchet, environment)
shoot_distance(precise_trebuchet, environment)

Now she can include MelissasModule.jl once, and change and include MelissasCode.jl as often as she wants. But what if she wants to make changes to the module? If she changes the code in the module, re-includes the module and runs her code again, she only gets a bunch of warnings, but her changes are not applied.

Revise.jl

Revise.jl is a package that can keep track of changes in your files and load these in a running Julia session.

Melissa needs to take two things into account:

Thus she now runs

using Revise

includet("MelissasModule.jl")

include("MelissasCode.jl")
100.0975848073789

and any change she makes in MelissasModule.jl will be visible in the next run of her code.

Did I say any changes?

Well, almost any. Revise can’t track changes to structures.

Key Points

  • Modules introduce namespaces

  • Public API has to be documented and can be exported


Creating Packages

Overview

Teaching: 15 min
Exercises: 15 min
Questions
  • How to create a package?

Objectives
  • Learn setting up a project using modules.

  • Learn common package structure.

  • Learn to browse GitHub or juliahub for packages and find documentation.

Melissa is now confident that her module fine and she wants to make it available for the rest of her physics club. She decides to put it in a package. This way she can also locally use Julia’s package manager for managing her module.

From project to package

The path from having a module to having a package is actually very short: Packages need a name and a uuid field in their Project.toml.

A UUID is a universally unique identifier. Thankfully Julia comes with the UUIDs package, that can generate uuids for Melissa via UUIDs.uuid4().

In addition Melissa needs to have a specific directory structure. She looks at the example package Example.jl which has the following structure:

├── docs
│   ├── make.jl
│   ├── Project.toml
│   └── src
│       └── index.md
├── LICENSE.md
├── Project.toml
├── README.md
├── src
│   └── Example.jl
└── test
    └── runtests.jl

Make it a package

Open your Project.toml and add name = <your name>, uuid = <your uuid> and optionally an authors field, each on a separate line.

Now Melissa can use

pkg> dev . # or path to package instead of `.`

instead of needing to includet MelissasModule.jl, and she can write using MelissasModule instead of .using MelissasModule.

Register a package

In order for her friends to be able to get the package, Melissa registers the package in the general registry. This can be done either via JuliaHub or by making a pull request on GitHub which can also be automated by the Julia Registrator.

Creating a new package

Melissa thinks next time she will start with a package right away.

Browsing the packages she found PkgTemplates.jl and PkgSkeleton.jl which makes setting up the typical folder structure very easy.

Create your own package

Look at the documentation of the package creation helper packages and create a new package using generate.

Key Points

  • The general registry is hosted on GitHub.

  • Packaging is easy


Adding tests

Overview

Teaching: 10 min
Exercises: 30 min
Questions
  • What are unit tests?

  • How are tests organized in Julia?

Objectives
  • Learn to create unit tests and test-sets using the Test standard library

Unit tests

Now that Melissa has released her first package she fears that future changes will impact the existing functionality of her package. This can be prevented by adding tests to her package.

Looking at the structure of other packages Melissa figured out that tests usually go in a separate test folder next to the src folder. This should contain a runtests.jl file.

The standard library Test provides the functionality for writing tests: namely, the macros @test and @testset.

@test can be used to test a single equality, such as

using Test
@test 1 + 1 == 2

Several tests can be grouped in a test-set with a descriptive name

using Test
@testset "Test arithmetic equalities" begin
    @test 1 + 1 == 2
end

With this Melissa can run her test using the pkg mode of the REPL:

(MelissasModule) pkg> test

Test specific dependencies

Melissa needed to add Test to her package in order to run the code above, but actually Test is not needed for her package other than testing. Thus it is possible to move the Test entry in the Project.toml file from [deps] to an [extras] section and then add another entry:

[targets]
test = ["Test"]

Check out the sample project file for a complete example.

Create a test for MelissasModule

Create a test that ensures that shoot_distance returns a value that is between target - ε and target + ε.

Solution

using MelissasModule
using Test

@testset "Test hitting target" begin
    imprecise_trebuchet = Trebuchet(500.0, 0.25pi)
    environment = Environment(5, 100)
    precise_trebuchet = aim(imprecise_trebuchet, environment)
    @test 100 - 0.1 <= shoot_distance(precise_trebuchet, environment) <= 100 + 0.1
    # default ε is 0.1
end

Key Points

  • Tests are important