Exploring the Concept of Limits

Introduction

Nowadays, knowledge of machine learning is very important for a software engineer, but machine learning is not a fundamental subject of mathematics like Calculus or Linear Algebra.

If you know PyTorch, one of the machine learning frameworks, you will read the Calculus, Linear Algebra and Probability in the official document, e.g. How does PyTorch compute the conjugate Wirtinger derivative, torch.linalg.qr and ExponentialFamily

So, I decide to write a series of blog posts called 'Before Machine Learning', forcing to summarise the key concepts.

Definition

• Definition of $\lim\limits_{x \to a} f(x) = L$
  
  $\forall$ $\epsilon > 0$, $\exist$ $\delta > 0$ $\ni$ if $0 < |x - a| < \delta$ then $|f(x) - L| < \epsilon$

• Definition of $\lim\limits_{(x,\ y) \to (a,\ b)} f(x,\ y) = L$
  
  $\forall$ $\epsilon > 0$, $\exist$ $\delta > 0$ $\ni$ if $(x,\ y)$ $\in$ D and $0 < \sqrt{(x - a)^2 + (y - b)^2} < \delta$ then $|f(x,\ y) - L| < \epsilon$, D is a Domain of $f(x,\ y)$
  
  

Limit of Function

• $\lim\limits_{x \to a^+} f(x) = L_1$, $\lim\limits_{x \to a^-} f(x) = L_2$, $f(a) = L_0$ (or $f(a)$ doesn't exist)
  
  

• $\lim\limits_{x \to a^-} f(x) = \lim\limits_{x \to a^+} f(x) = \lim\limits_{x \to a} f(x) = L_3$, $f(a) \neq L_3$
  
  

• $\lim\limits_{x \to a^-} f(x) = \lim\limits_{x \to a^+} f(x) = \lim\limits_{x \to a} f(x) = f(a) = L_4$
  
  

Limit Laws

• $\lim\limits_{x \to a} (c \cdot f(x)) = c \cdot \lim\limits_{x \to a} f(x)$, $c$ is a scalar.
  
  
• $\lim\limits_{x \to a} (f(x) + g(x)) = \lim\limits_{x \to a} f(x) + \lim\limits_{x \to a} g(x)$
  
  
• $\lim\limits_{x \to a} (f(x) - g(x)) = \lim\limits_{x \to a} f(x) - \lim\limits_{x \to a} g(x)$
  
  
• $\lim\limits_{x \to a} (f(x) \cdot g(x)) = \lim\limits_{x \to a} f(x) \cdot \lim\limits_{x \to a} g(x)$
  
  
• $\lim\limits_{x \to a} \frac{f(x)}{g(x)} = \frac{\lim\limits_{x \to a} f(x)}{\lim\limits_{x \to a} g(x)}$, $\lim\limits_{x \to a} g(x) \neq 0$
  
  

Continuity

$f(x,\ y)$ is continuous at every point $(a,\ b)$ in the domain if $\lim\limits_{(x,\ y) \to (a,\ b)} f(x,\ y) = f(a,\ b)$

Asymptotes

• Horizontal Asymptotes
  
  $y = L$ if $\lim\limits_{x \to \infty} f(x) = L$ or $\lim\limits_{x \to -\infty} f(x) = L$
  
  
• Vertical Asymptotes
  
  $x = L$ if $\lim\limits_{x \to L^\pm} f(x) = \pm\infty$
  
  
• Slant Asymptotes
  
  $y = mx + b$ if $\lim\limits_{x \to \infty} (f(x) - (mx + b)) = 0$ or $\lim\limits_{x \to -\infty} (f(x) - (mx + b)) = 0$

The Sandwich Theorem

$\forall x \to a$, if $f(x) \le h(x) \le g(x)$ and $\lim\limits_{x \to a} f(x) = \lim\limits_{x \to a} g(x) = L$, then $\lim\limits_{x \to a} h(x) = L$

The Intermediate Value Theorem

If $f$ is continuous on $[a,\ b]$, $f(a) < N < f(b)$ or $f(b) < N < f(a)$, then $\exist$ c $\in (a,\ b) \ni f(c) = N$

Derivative

• Derivative of a function $f(a)$
  
  $f'(a) = \lim\limits_{h \to 0} \frac{f(a + h) - f(a)}{h}$

• Derivative of a function $f(a,\ b)$
  
  $f_x(a,\ b) = \lim\limits_{h \to 0} \frac{f(a + h,\ b) - f(a,\ b)}{h}$

  $f_y(a,\ b) = \lim\limits_{h \to 0} \frac{f(a,\ b + h) - f(a,\ b)}{h}$

Conclusion

In this post you know about the relationship between limit and derivative. It is one of the key concepts in calculus. Thank you for reading and see you soon.