Differential calculus in higher dimension ​

In this part of the course we work on the following skills:

• Become comfortable working with coordinates in arbitrary dimension.
• Develop an intuition for working with vector fields.
• Understand the subtleties of derivatives in dimension greater than 1, evaluate and manipulate partial derivatives, directional derivatives, Jacobian.

Here we start to consider higher dimensional space. That is, instead of $\mathbb{R}$ we consider ${\mathbb{R}}^{n}$ for $n\in \mathbb{N}$. We will particularly focus on 2D and 3D but everything also holds in any dimension. Going beyond $\mathbb{R}$ we have more options for functions and correspondingly more options for derivatives. Various different notation is commonly used. Here we will primarily use $\left(x,y\right)\in {\mathbb{R}}^{2}$, $\left(x,y,z\right)\in {\mathbb{R}}^{3}$ or, more generally, $\mathbf{x}=\left({x}_{1},{x}_{2},\dots ,{x}_{n}\right)\in {\mathbb{R}}^{n}$ where ${x}_{1}\in \mathbb{R},\dots ,{x}_{n}\in \mathbb{R}$. For example, ${\mathbb{R}}^{2}$ is the plane, ${\mathbb{R}}^{3}$ is 3D space.

Definition (inner product)

$\mathbf{x}\cdot \mathbf{y}=\sum _{k=1}^{n}{x}_{k}{y}_{k}\in \mathbb{R}$

We recall that the inner product being zero has a geometric meaning, it means that the two vectors are orthogonal. We also recall that the "length" of a vector is given by the norm, defined as follows.

Definition (norm)

$‖\mathbf{x}‖=\sqrt{\mathbf{x}\cdot \mathbf{x}}={\left(\sum _{k=1}^{n}{x}_{k}^{2}\right)}^{\frac{1}{2}}$.

For example, in ${\mathbb{R}}^{2}$ then $‖\left(x,y\right)‖=\sqrt{{x}^{2}+{y}^{2}}$. There are various convenient properties for working with norms and inner products, in particular, the Cauchy-Schwarz inequality and the triangle inequality $‖\mathbf{x}+\mathbf{y}‖\le ‖\mathbf{x}‖+‖\mathbf{y}‖$.

The primary higher-dimensional functions we consider in this course are:

• Scalar fields: $f:{\mathbb{R}}^{n}\to \mathbb{R}$
• Vector fields: $\mathbf{F}:{\mathbb{R}}^{n}\to {\mathbb{R}}^{n}$
• Paths: $\mathbit{\alpha }:\mathbb{R}\to {\mathbb{R}}^{n}$
• Change of coordinates: $\mathbf{x}:{\mathbb{R}}^{n}\to {\mathbb{R}}^{n}$

These possibilities all fit into the general pattern of $f:{\mathbb{R}}^{n}\to {\mathbb{R}}^{m}$ for $n,m\in \mathbb{N}$ but tradition and use of the function gives us different terminology and symbols. Such functions are useful for representing various practical things, for example: gravitational force; temperature in a region; wind velocity; fluid flow; electric field; etc.

Open sets, closed sets, boundary, continuity ​

Let $\mathbf{a}\in {\mathbb{R}}^{n}$, $r>0$. The open $n$-ball of radius $r$ and centre $\mathbf{a}$ is written as

$B\left(\mathbf{a},r\right):=\left\{\mathbf{x}\in {\mathbb{R}}^{n}:‖\mathbf{x}-\mathbf{a}‖

Definition (interior point)

Let $S\subset {\mathbb{R}}^{n}$. A point $\mathbf{a}\in S$ is said to be an interior point if there is $r>0$ such that $B\left(\mathbf{a},r\right)\subset S$. The set of all interior points of $S$ is denoted $\mathrm{int}S$.

Definition (open set)

A set $S\subset {\mathbb{R}}^{n}$ is said to be open if all of its points are interior points, i.e., if $\mathrm{int}S=S$.

For example, open intervals, open disks, open balls, unions of open intervals, etc., are all open sets.

Lemma

Let $r>0$, $\mathbf{a}\in {\mathbb{R}}^{n}$. The set $B\left(\mathbf{a},r\right)\subset {\mathbb{R}}^{n}$ is open.

Proof

Let $\mathbf{b}\in B\left(\mathbf{a},r\right)$. It suffices to show that $\mathbf{b}$ is an interior point. (1) Let ${r}_{1}=\parallel \mathbf{b}-\mathbf{a}\parallel . (2) Let ${r}_{2}=\left(r-{r}_{1}\right)/2$. (3) We claim that $B\left(\mathbf{b},{r}_{2}\right)\subset B\left(\mathbf{a},r\right)$: In order to see this take any $\mathbf{c}\in B\left(\mathbf{b},{r}_{2}\right)$ and observe that

$\parallel \mathbf{c}-\mathbf{a}\parallel \le \parallel \mathbf{c}-\mathbf{b}\parallel +\parallel \mathbf{b}-\mathbf{a}\parallel \le {r}_{2}+{r}_{1}=\frac{r+{r}_{1}}{2}

Observe that the radius of the ball will be small for points close to the boundary.

Definition (Cartesian product)

If ${A}_{1}\subset \mathbb{R}$, ${A}_{2}\subset \mathbb{R}$ then the Cartesian product is defined as

${A}_{1}×{A}_{2}:=\left\{\left(x,y\right):x\in {A}_{1},y\in {A}_{2}\right\}\subset {\mathbb{R}}^{2}.$

Analogously the Cartesian product can be defined in higher dimensions: If ${A}_{1}\subset {\mathbb{R}}^{m}$, ${A}_{2}\subset {\mathbb{R}}^{n}$ then the Cartesian product ${A}_{1}×{A}_{2}$ is defined as the set of all points $\left({x}_{1},\dots ,{x}_{m},{y}_{1},\dots ,{y}_{n}\right)\in {\mathbb{R}}^{m+n}$ such that $\left({x}_{1},\dots ,{x}_{m}\right)\in {A}_{1}$ and $\left({y}_{1},\dots ,{y}_{n}\right)\in {A}_{2}$.

Lemma

If ${A}_{1},{A}_{2}$ are open subsets of $\mathbb{R}$ then ${A}_{1}×{A}_{2}$ is an open subset of ${\mathbb{R}}^{2}$.

Proof

Let $\mathbf{a}=\left({a}_{1},{a}_{2}\right)\in {A}_{1}×{A}_{2}\subset {\mathbb{R}}^{2}$. Since ${A}_{1}$ is open there exists ${r}_{1}>0$ such that $B\left({a}_{1},{r}_{1}\right)\subset {A}_{1}$. Similarly for ${A}_{2}$. Let $r=min\left\{{r}_{1},{r}_{2}\right\}$. This all means that $B\left(\mathbf{a},r\right)\subset B\left({a}_{1},{r}_{1}\right)×B\left({a}_{2},{r}_{2}\right)\subset {A}_{1}×{A}_{2}$.

Discussing the "interior" of the set naturally suggests the topic of the "boundary" of the set. In the following definitions we develop this idea.

Definition (exterior points)

Let $S\subset {\mathbb{R}}^{n}$. A point $\mathbf{a}\notin S$ is said to be an exterior point if there exists $r>0$ such that $B\left(\mathbf{a},r\right)\cap S=\mathrm{\varnothing }$. The set of all exterior points of $S$ is denoted $\mathrm{ext}S$.

Observe that $\mathrm{ext}S$ is an open set. We use the notation ${S}^{c}={\mathbb{R}}^{n}\setminus S$ and we say that ${C}^{c}$ is the complement of the set $S$.

Definition (boundary)

The set ${\mathbb{R}}^{n}\setminus \left(\mathrm{int}S\cup \mathrm{ext}S\right)$ is called the boundary of $S\subset {\mathbb{R}}^{n}$ and is denoted $\partial S$.

Definition (closed)

A set $S\subset {\mathbb{R}}^{n}$ is said to be closed if $\partial S\subset S$.

Lemma

$S$ is open $⟺$ ${S}^{c}$ is closed.

Proof

Observe that ${\mathbb{R}}^{n}=\mathrm{int}S\cup \partial S\cup \mathrm{ext}S$ (disjointly). If $\mathbf{x}\in \partial S$ then, for every $r>0$, $B\left(\mathbf{x},r\right)\cap S\ne \mathrm{\varnothing }$ and so $\mathbf{x}\in \partial \left({S}^{c}\right)$. Similarly with $S$ and ${S}^{c}$ swapped and so $\partial S=\partial \left({S}^{c}\right)$. If $S$ is open then $\mathrm{int}S=S$ and ${S}^{c}=\mathrm{ext}S\cup \partial S=\mathrm{ext}S\cup \partial \left({S}^{c}\right)$ and so ${S}^{c}$ is closed. If $S$ is not open then there exists $\mathbf{a}\in \partial S\cap S$. Additionally $\mathbf{a}\in \partial \left({S}^{c}\right)\cap S$ hence ${S}^{c}$ is not closed.

Limits and continuity ​

Let $S\subset {\mathbb{R}}^{n}$ and $\mathbf{f}:S\to {\mathbb{R}}^{m}$. If $\mathbf{a}\in {\mathbb{R}}^{n}$, $\mathbf{b}\in {\mathbb{R}}^{m}$ we write $\underset{\mathbf{x}\to \mathbf{a}}{lim}\mathbf{f}\left(\mathbf{x}\right)=\mathbf{b}$ to mean that $‖\mathbf{f}\left(\mathbf{x}\right)-\mathbf{b}‖\to 0$ as $‖\mathbf{x}-\mathbf{a}‖\to 0$. Observe how, if $n=m=1$, this is the familiar notion of continuity for functions on $\mathbb{R}$.

Definition (Continuous)

A function $\mathbf{f}$ is said to be continuous at $\mathbf{a}$ if $\mathbf{f}$ is defined at $\mathbf{a}$ and $\underset{\mathbf{x}\to \mathbf{a}}{lim}\mathbf{f}\left(\mathbf{x}\right)=\mathbf{f}\left(\mathbf{a}\right)$. We say $\mathbf{f}$ is continuous on $S$ if $\mathbf{f}$ is continuous at each point of $S$.

Even functions which look "nice" can fail to be continuous as we can see in the following example.

Example (continuity in higher dimensions)

Let $f$ be defined, for $\left(x,y\right)\ne \left(0,0\right)$, as

$f\left(x,y\right)=\frac{xy}{{x}^{2}+{y}^{2}}$

and $f\left(0,0\right)=0$. What is the behaviour of $f$ when approaching $\left(0,0\right)$ along the following lines?

linevalue
$\left\{x=0\right\}$$f\left(0,t\right)=0$
$\left\{y=0\right\}$$f\left(t,0\right)=0$
$\left\{x=y\right\}$$f\left(t,t\right)=\frac{1}{2}$
$\left\{x=-y\right\}$$f\left(t,t\right)=-\frac{1}{2}$

Theorem

Suppose that $\underset{\mathbf{x}\to \mathbf{a}}{lim}\mathbf{f}\left(\mathbf{x}\right)=\mathbf{b}$ and $\underset{\mathbf{x}\to \mathbf{a}}{lim}\mathbf{g}\left(\mathbf{x}\right)=\mathbf{c}$. Then

1. $\underset{\mathbf{x}\to \mathbf{a}}{lim}\left(\mathbf{f}\left(\mathbf{x}\right)+\mathbf{g}\left(\mathbf{x}\right)\right)=\mathbf{b}+\mathbf{c}$,
2. $\underset{\mathbf{x}\to \mathbf{a}}{lim}\lambda \mathbf{f}\left(\mathbf{x}\right)=\lambda \mathbf{b}$ for every $\lambda \in \mathbb{R}$,
3. $\underset{\mathbf{x}\to \mathbf{a}}{lim}\mathbf{f}\left(\mathbf{x}\right)\cdot \mathbf{g}\left(\mathbf{x}\right)=\mathbf{b}\cdot \mathbf{c}$,
4. $\underset{\mathbf{x}\to \mathbf{a}}{lim}‖\mathbf{f}\left(\mathbf{x}\right)‖=‖\mathbf{b}‖$.

We prove a couple of the parts of the above theorem here, the other parts are left as exercises.

Proof of part 3.

Observe that $\mathbf{f}\left(\mathbf{x}\right)\cdot \mathbf{g}\left(\mathbf{x}\right)-\mathbf{b}\cdot \mathbf{c}=\left(\mathbf{f}\left(\mathbf{x}\right)-\mathbf{b}\right)\cdot \left(\mathbf{g}\left(\mathbf{x}\right)-\mathbf{c}\right)+\mathbf{b}\cdot \left(\mathbf{g}\left(\mathbf{x}\right)-\mathbf{c}\right)+\mathbf{c}\cdot \left(\mathbf{f}\left(\mathbf{x}\right)-\mathbf{b}\right)$. By the triangle inequality and Cauchy-Schwarz,

$\begin{array}{rl}‖\mathbf{f}\left(\mathbf{x}\right)\cdot \mathbf{g}\left(\mathbf{x}\right)-\mathbf{b}\cdot \mathbf{c}‖& \le ‖\mathbf{f}\left(\mathbf{x}\right)-\mathbf{b}‖‖\mathbf{g}\left(\mathbf{x}\right)-\mathbf{c}‖\\ & \phantom{\rule{1em}{0ex}}+‖\mathbf{b}‖‖\mathbf{g}\left(\mathbf{x}\right)-\mathbf{c}‖\\ & \phantom{\rule{1em}{0ex}}+‖\mathbf{c}‖‖\mathbf{f}\left(\mathbf{x}\right)-\mathbf{b}‖.\end{array}$

Since we already know that $‖\mathbf{f}\left(\mathbf{x}\right)-\mathbf{b}‖\to 0$ and $‖\mathbf{g}\left(\mathbf{x}\right)-\mathbf{c}‖\to 0$ as $\mathbf{x}\to \mathbf{a}$, this implies that $‖\mathbf{f}\left(\mathbf{x}\right)\cdot \mathbf{g}\left(\mathbf{x}\right)-\mathbf{b}\cdot \mathbf{c}‖\to 0$.

Proof of part 4.

Take $\mathbf{f}=\mathbf{g}$ in part (c) implies that $\underset{\mathbf{x}\to \mathbf{a}}{lim}{‖\mathbf{f}\left(\mathbf{x}\right)‖}^{2}={‖\mathbf{b}‖}^{2}$.

When writing a vector field (or similar functions) it is often convenient to divide the higher-dimensional function into smaller parts. We call these parts the components of a vector field. For example $\mathbf{F}\left(\mathbf{x}\right)=\left({F}_{1}\left(\mathbf{x}\right),{F}_{2}\left(\mathbf{x}\right)\right)$ in 2D, $\mathbf{F}\left(\mathbf{x}\right)=\left({F}_{1}\left(\mathbf{x}\right),{F}_{2}\left(\mathbf{x}\right),{F}_{3}\left(\mathbf{x}\right)\right)$ in 3D, etc.

Theorem

Let $\mathbf{F}\left(\mathbf{x}\right)=\left({F}_{1}\left(\mathbf{x}\right),{F}_{2}\left(\mathbf{x}\right)\right)$. Then $\mathbf{F}$ is continuous if and only if ${F}_{1}$ and ${F}_{2}$ are continuous.

Proof

We will independently prove the two implications.

• ($⇒$) Let ${\mathbf{e}}_{1}=\left(1,0\right)$, ${\mathbf{e}}_{2}=\left(0,1\right)$ and observe that ${F}_{k}\left(\mathbf{x}\right)=\mathbf{F}\left(\mathbf{x}\right)\cdot {\mathbf{e}}_{k}$. We have already shown that the continuity of two vector fields implies the continuity of the inner product.
• ($⇐$) By definition of the norm${‖\mathbf{F}\left(\mathbf{x}\right)-\mathbf{F}\left(\mathbf{a}\right)‖}^{2}=\sum _{k=1}^{2}{\left({F}_{k}\left(\mathbf{x}\right)-{F}_{k}\left(\mathbf{a}\right)\right)}^{2}$and we know $‖{F}_{k}\left(\mathbf{x}\right)-{F}_{k}\left(\mathbf{a}\right)‖\to 0$ as $‖\mathbf{x}-\mathbf{a}‖\to 0$.

In higher dimensions the analogous statement is true for the vector field $\mathbf{F}\left(\mathbf{x}\right)=\left({F}_{1}\left(\mathbf{x}\right),\dots ,{F}_{m}\left(\mathbf{x}\right)\right)$ with exactly the same proof. I.e., $\mathbf{F}$ is continuous if and only if each ${f}_{k}$ is continuous.

Example (polynomials)

A polynomial in $n$ variables is a scalar field on ${\mathbb{R}}^{n}$ of the form

$f\left({x}_{1},\dots ,{x}_{n}\right)=\sum _{{k}_{1}=0}^{j}\cdots \sum _{{k}_{n}=0}^{j}{c}_{{k}_{1},\dots ,{k}_{n}}{x}_{1}^{{k}_{1}}\cdots {x}_{n}^{{k}_{n}}.$

E.g., $f\left(x,y\right):=x+2xy-{x}^{2}$ is a polynomial in $2$ variables. Polynomials are continuous everywhere in ${\mathbb{R}}^{n}$. This is because they are the finite sum of products of continuous scalar fields.

Example (rational functions)

A rational function is a scalar field

$f\left(\mathbf{x}\right)=\frac{p\left(\mathbf{x}\right)}{q\left(\mathbf{x}\right)}$

where $p\left(\mathbf{x}\right)$ and $q\left(\mathbf{x}\right)$ are polynomials. A rational function is continuous at every point $\mathbf{x}$ such that $q\left(\mathbf{x}\right)\ne 0$.

As described in the following result, the continuity of functions continues to hold, in an intuitive way, under composition of functions.

Theorem

Suppose $S\subset {\mathbb{R}}^{l}$, $T\subset {\mathbb{R}}^{m}$, $\mathbf{f}:S\to {\mathbb{R}}^{m}$, $\mathbf{g}:T\to {\mathbb{R}}^{n}$ and that $\mathbf{f}\left(S\right)\subset T$ so that

$\left(\mathbf{g}\circ \mathbf{f}\right)\left(\mathbf{x}\right)=\mathbf{g}\left(\mathbf{f}\left(\mathbf{x}\right)\right)$

makes sense. If $\mathbf{f}$ is continuous at $\mathbf{a}\in S$ and $\mathbf{g}$ is continuous at $\mathbf{f}\left(\mathbf{a}\right)$ then $\mathbf{g}\circ \mathbf{f}$ is continuous at $\mathbf{a}$.

Proof

$lim\mathbf{x}\to \mathbf{a}‖\mathbf{f}\left(\mathbf{g}\left(\mathbf{x}\right)\right)-\mathbf{f}\left(\mathbf{g}\left(\mathbf{a}\right)\right)‖=lim\mathbf{y}\to \mathbf{g}\left(\mathbf{a}\right)‖\mathbf{f}\left(\mathbf{y}\right)-\mathbf{f}\left(\mathbf{g}\left(\mathbf{a}\right)\right)‖=0$

Example

We can consider the scalar field