1 Functions and Operations

Topics Dimensionality; Interval Notation for \({\bf R}^1\); Neighborhoods: Intervals, Disks, and Balls; Introduction to Functions; Domain and Range; Some General Types of Functions; \(\log\), \(\ln\), and \(\exp\); Other Useful Functions; Graphing Functions; Solving for Variables; Finding Roots; Limit of a Function; Continuity; Sets, Sets, and More Sets.

1.1 Summation Operators \(\sum\) and \(\prod\)

Addition (+), Subtraction (-), multiplication and division are basic operations of arithmetic – combining numbers. In statistics and calculus, we want to add a sequence of numbers that can be expressed as a pattern without needing to write down all its components. For example, how would we express the sum of all numbers from 1 to 100 without writing a hundred numbers?

For this we use the summation operator \(\sum\) and the product operator \(\prod\).

Summation:

\[\sum\limits_{i=1}^{100} x_i = x_1+x_2+x_3+\cdots+x_{100}\]

The bottom of the \(\sum\) symbol indicates an index (here, \(i\)), and its start value \(1\). At the top is where the index ends. The notion of “addition” is part of the \(\sum\) symbol. The content to the right of the summation is the meat of what we add. While you can pick your favorite index, start, and end values, the content must also have the index.

\(\sum\limits_{i=1}^n c x_i = c \sum\limits_{i=1}^n x_i\)
\(\sum\limits_{i=1}^n (x_i + y_i) = \sum\limits_{i=1}^n x_i + \sum\limits_{i=1}^n y_i\)
\(\sum\limits_{i=1}^n c = n c\)

Product:

\[\prod\limits_{i=1}^n x_i = x_1 x_2 x_3 \cdots x_n\]

Properties:

\(\prod\limits_{i=1}^n c x_i = c^n \prod\limits_{i=1}^n x_i\)
\(\prod\limits_{i=k}^n c x_i = c^{n-k+1} \prod\limits_{i=k}^n x_i\)
\(\prod\limits_{i=1}^n (x_i + y_i) =\) a total mess
\(\prod\limits_{i=1}^n c = c^n\)

Other Useful Functions

Factorials!:

\[x! = x\cdot (x-1) \cdot (x-2) \cdots (1)\]

Modulo: Tells you the remainder when you divide the first number by the second.

\(17\mod3 = 2\)
\(100 \ \% \ 30 = 10\)

Example 1.1 (Operators)

\(\sum\limits_{i=1}^{5} i =\)
\(\prod\limits_{i=1}^{5} i =\)
\(14 \mod 4 =\)
\(4! =\)

Exercise 1.1 (Operators) Let \(x_1 = 4, x_2 = 3, x_3 = 7, x_4 = 11, x_5 = 2\)

\(\sum\limits_{i=1}^{3} (7)x_i\)
\(\sum\limits_{i=1}^{5} 2\)
\(\prod\limits_{i=3}^{5} (2)x_i\)

1.2 Introduction to Functions

A function (in \({\bf R}^1\)) is a mapping, or transformation, that relates members of one set to members of another set. For instance, if you have two sets: set \(A\) and set \(B\), a function from \(A\) to \(B\) maps every value \(a\) in set \(A\) such that \(f(a) \in B\). Functions can be “many-to-one”, where many values or combinations of values from set \(A\) produce a single output in set \(B\), or they can be “one-to-one”, where each value in set \(A\) corresponds to a single value in set \(B\). A function by definition has a single function value for each element of its domain. This means, there cannot be “one-to-many” mapping.

Dimensionality: \({\bf R}^1\) is the set of all real numbers extending from \(-\infty\) to \(+\infty\) — i.e., the real number line. \({\bf R}^n\) is an \(n\)-dimensional space, where each of the \(n\) axes extends from \(-\infty\) to \(+\infty\).

\({\bf R}^1\) is a one dimensional line.
\({\bf R}^2\) is a two dimensional plane.
\({\bf R}^3\) is a three dimensional space.

Points in \({\bf R}^n\) are ordered \(n\)-tuples (just means an combination of \(n\) elements where order matters), where each element of the \(n\)-tuple represents the coordinate along that dimension.

For example:

\({\bf R}^1\): (3)
\({\bf R}^2\): (-15, 5)
\({\bf R}^3\): (86, 4, 0)

Examples of mapping notation:

Function of one variable: \(f:{\bf R}^1\to{\bf R}^1\)

\(f(x)=x+1\). For each \(x\) in \({\bf R}^1\), \(f(x)\) assigns the number \(x+1\).

Function of two variables: \(f: {\bf R}^2\to{\bf R}^1\).

\(f(x,y)=x^2+y^2\). For each ordered pair \((x,y)\) in \({\bf R}^2\), \(f(x,y)\) assigns the number \(x^2+y^2\).

We often use variable \(x\) as input and another \(y\) as output, e.g. \(y=x+1\)

Example 1.2 (Functions) For each of the following, state whether they are one-to-one or many-to-one functions.

For \(x \in [0,\infty]\), \(f : x \rightarrow x^2\) (this could also be written as \(f(x) = x^2\)).
For \(x \in [-\infty, \infty]\), \(f: x \rightarrow x^2\).

Exercise 1.2 (Functions) For each of the following, state whether they are one-to-one or many-to-one functions.

For \(x \in [-3, \infty]\), \(f: x \rightarrow x^2\).
For \(x \in [0, \infty]\), \(f: x \rightarrow \sqrt{x}\)

Some functions are defined only on proper subsets of \({\bf R}^n\).

Domain: the set of numbers in \(X\) at which \(f(x)\) is defined.
Range: elements of \(Y\) assigned by \(f(x)\) to elements of \(X\), or \[f(X)=\{ y : y=f(x), x\in X\}\] Most often used when talking about a function \(f:{\bf R}^1\to{\bf R}^1\).
Image: same as range, but more often used when talking about a function \(f:{\bf R}^n\to{\bf R}^1\).

Some General Types of Functions

Monomials: \(f(x)=a x^k\)

\(a\) is the coefficient. \(k\) is the degree.

Examples: \(y=x^2\), \(y=-\frac{1}{2}x^3\)

Polynomials: sum of monomials.

Examples: \(y=-\frac{1}{2}x^3+x^2\), \(y=3x+5\)

The degree of a polynomial is the highest degree of its monomial terms. Also, it’s often a good idea to write polynomials with terms in decreasing degree.

Exponential Functions: Example: \(y=2^x\)

1.3 \(\log\) and \(\exp\)

Relationship of logarithmic and exponential functions: \[y=\log_a(x) \iff a^y=x\]

The log function can be thought of as an inverse for exponential functions. \(a\) is referred to as the “base” of the logarithm.

Common Bases: The two most common logarithms are base 10 and base \(e\).

Base 10: \(\quad y=\log_{10}(x) \iff 10^y=x\). The base 10 logarithm is often simply written as “\(\log(x)\)” with no base denoted.
Base \(e\): \(\quad y=\log_e(x) \iff e^y=x\). The base \(e\) logarithm is referred to as the “natural” logarithm and is written as ``\(\ln(x)\)“.

Properties of exponential functions:

\(a^x a^y = a^{x+y}\)
\(a^{-x} = 1/a^x\)
\(a^x/a^y = a^{x-y}\)
\((a^x)^y = a^{x y}\)
\(a^0 = 1\)

Properties of logarithmic functions (any base):

Generally, when statisticians or social scientists write \(\log(x)\) they mean \(\log_e(x)\). In other words: \(\log_e(x) \equiv \ln(x) \equiv \log(x)\)

\[\log_a(a^x)=x\] and \[a^{\log_a(x)}=x\]

\(\log(x y)=\log(x)+\log(y)\)
\(\log(x^y)=y\log(x)\)
\(\log(1/x)=\log(x^{-1})=-\log(x)\)
\(\log(x/y)=\log(x\cdot y^{-1})=\log(x)+\log(y^{-1})=\log(x)-\log(y)\)
\(\log(1)=\log(e^0)=0\)

Change of Base Formula: Use the change of base formula to switch bases as necessary: \[\log_b(x) = \frac{\log_a(x)}{\log_a(b)}\]

Example: \[\log_{10}(x) = \frac{\ln(x)}{\ln(10)}\]

You can use logs to go between sum and product notation. This will be particularly important when you’re learning maximum likelihood estimation.

\[\begin{eqnarray*} \log \bigg(\prod\limits_{i=1}^n x_i \bigg) &=& \log(x_1 \cdot x_2 \cdot x_3 \cdots \cdot x_n)\\ &=& \log(x_1) + \log(x_2) + \log(x_3) + \cdots + \log(x_n)\\ &=& \sum\limits_{i=1}^n \log (x_i) \end{eqnarray*}\]

Therefore, you can see that the log of a product is equal to the sum of the logs. We can write this more generally by adding in a constant, \(c\):

\[\begin{eqnarray*} \log \bigg(\prod\limits_{i=1}^n c x_i\bigg) &=& \log(cx_1 \cdot cx_2 \cdots cx_n)\\ &=& \log(c^n \cdot x_1 \cdot x_2 \cdots x_n)\\ &=& \log(c^n) + \log(x_1) + \log(x_2) + \cdots + \log(x_n)\\\\ &=& n \log(c) + \sum\limits_{i=1}^n \log (x_i)\\ \end{eqnarray*}\]

Example 1.3 (Logarithmic Functions) Evaluate each of the following logarithms

\(\log_4(16)\)
\(\log_2(16)\)

Simplify the following logarithm. By “simplify”, we actually really mean - use as many of the logarithmic properties as you can.

\(\log_4(x^3y^5)\)

Exercise 1.3 (Logarithmic Functions) Evaluate each of the following logarithms

\(\log_\frac{3}{2}(\frac{27}{8})\)

Simplify each of the following logarithms. By “simplify”, we actually really mean - use as many of the logarithmic properties as you can.

\(\log(\frac{x^9y^5}{z^3})\)
\(\ln{\sqrt{xy}}\)

1.4 Graphing Functions

What can a graph tell you about a function?

Is the function increasing or decreasing? Over what part of the domain?
How ``fast” does it increase or decrease?
Are there global or local maxima and minima? Where?
Are there inflection points?
Is the function continuous?
Is the function differentiable?
Does the function tend to some limit?
Other questions related to the substance of the problem at hand.

1.5 Solving for Variables and Finding Roots

Sometimes we’re given a function \(y=f(x)\) and we want to find how \(x\) varies as a function of \(y\). Use algebra to move \(x\) to the left hand side (LHS) of the equation and so that the right hand side (RHS) is only a function of \(y\).

Example 1.4 (Solving for Variables) Solve for x:

\(y=3x+2\)
\(y=e^x\)

Solving for variables is especially important when we want to find the roots of an equation: those values of variables that cause an equation to equal zero. Especially important in finding equilibria and in doing maximum likelihood estimation.

Procedure: Given \(y=f(x)\), set \(f(x)=0\). Solve for \(x\).

Multiple Roots: \[f(x)=x^2 - 9 \quad\Longrightarrow\quad 0=x^2 - 9 \quad\Longrightarrow\quad 9=x^2 \quad\Longrightarrow\quad \pm \sqrt{9}=\sqrt{x^2} \quad\Longrightarrow\quad \pm 3=x\]

Quadratic Formula: For quadratic equations \(ax^2+bx+c=0\), use the quadratic formula: \[x=\frac{-b\pm\sqrt{b^2-4ac}}{2a}\]

Exercise 1.4 (Finding Roots) Solve for x:

\(f(x)=3x+2 = 0\)
\(f(x)=x^2+3x-4=0\)
\(f(x)=e^{-x}-10 = 0\)

1.6 Sets

Interior Point: The point \(\bf x\) is an interior point of the set \(S\) if \(\bf x\) is in \(S\) and if there is some \(\epsilon\)-ball around \(\bf x\) that contains only points in \(S\). The interior of \(S\) is the collection of all interior points in \(S\). The interior can also be defined as the union of all open sets in \(S\).

If the set \(S\) is circular, the interior points are everything inside of the circle, but not on the circle’s rim.
Example: The interior of the set \(\{ (x,y) : x^2+y^2\le 4 \}\) is \(\{ (x,y) : x^2+y^2< 4 \}\) .

Boundary Point: The point \(\bf x\) is a boundary point of the set \(S\) if every \(\epsilon\)-ball around \(\bf x\) contains both points that are in \(S\) and points that are outside \(S\). The boundary is the collection of all boundary points.

If the set \(S\) is circular, the boundary points are everything on the circle’s rim.
Example: The boundary of \(\{ (x,y) : x^2+y^2\le 4 \}\) is \(\{ (x,y) : x^2+y^2 = 4 \}\).

Open: A set \(S\) is open if for each point \(\bf x\) in \(S\), there exists an open \(\epsilon\)-ball around \(\bf x\) completely contained in \(S\).

If the set \(S\) is circular and open, the points contained within the set get infinitely close to the circle’s rim, but do not touch it.
Example: \(\{ (x,y) : x^2+y^2<4 \}\)

Closed: A set \(S\) is closed if it contains all of its boundary points.

Alternatively: A set is closed if its complement is open.
If the set \(S\) is circular and closed, the set contains all points within the rim as well as the rim itself.
Example: \(\{ (x,y) : x^2+y^2\le 4 \}\)
Note: a set may be neither open nor closed. Example: \(\{ (x,y) : 2 < x^2+y^2\le 4 \}\)

Complement: The complement of set \(S\) is everything outside of \(S\).

If the set \(S\) is circular, the complement of \(S\) is everything outside of the circle.
Example: The complement of \(\{ (x,y) : x^2+y^2\le 4 \}\) is \(\{ (x,y) : x^2+y^2 > 4 \}\).

Empty: The empty (or null) set is a unique set that has no elements, denoted by {} or \(\emptyset\).

The empty set is an example of a set that is open and closed, or a “clopen” set.
Examples: The set of squares with 5 sides; the set of countries south of the South Pole.

Answers to Examples and Exercises

Answer to Example Example 1.1:

1 + 2 + 3 + 4 + 5 = 15
1 * 2 * 3 * 4 * 5 = 120
2
4 * 3 * 2 * 1 = 24

Answer to Exercise Exercise 1.1:

7(4 + 3 + 7) = 98
2 + 2 + 2 + 2 + 2 = 10
\(2^3(7)(11)(2)\) = 1232

Answer to Example Example 1.2:

one-to-one
many-to-one

Answer to Exercise Exercise 1.2:

many-to-one
one-to-one

Answer to Example Example 1.3:

2
4
\(3\log_4(x) + 5\log_4(y)\)

Answer to Exercise Exercise 1.3:

3
\(9\log(x) + 5\log(y) - 3\log(z)\)
\(\frac{1}{2}(\ln{x} + \ln{y})\)

Answer to Example Example 1.4:

\(y=3x+2 \quad\Longrightarrow\quad -3x=2-y \quad\Longrightarrow\quad 3x=y-2 \quad\Longrightarrow\quad x=\frac{1}{3}(y-2)\)
\(x = \ln{y}\)

Answer to Exercise Exercise 1.4:

\(\frac{-2}{3}\)
x = {1, -4}
x = - \(\ln10\)