[ML] Introduction to TensorFlow

TensorFlow is an end-to-end open source platform for machine learning. 

 

TensorFlow APIs are arranged hierarchically, with the high-level APIs built on the low-level APIs. Machine learning researchers use the low-level APIs to create and explore new machine learning algorithms. In this class, you will use a high-level API named tf.keras to define and train machine learning models and to make predictions. tf.keras is the TensorFlow variant of the open-source Keras API.

 

TensorFlow toolkit hierarchy

 

You'll use Colab as a programming environment. Colab is Google's version of Jupyter Notebook. 

 

 

🔑 정리하기

- 구글 Colab은 클라우드 프로그래밍 환경을 제공한다. 여기서 머신 러닝 오픈 API인 텐셀플로우도 사용 가능하다. 해당 강의에서는 텐셀 플로우의 다양한 계층에서도 가장 높은 계층인 tf.keras를 사용하는데, 사용을 위해서는 먼저 파이썬의 NumPy와 pandas에 대한 이해가 선행되어야 한다. NumPy는 배열 표현을 단순화하고 선형 함수 계산을 한다. pandas는 메모리에 있는 dataset을 쉽게 표현해준다.

 

Tensorflow와 Pytorch는 각각 구글과 페이스북에서 만든 딥러닝 프레임워크. Keras는 tf의 high-level api로, tf에 얹어 사용하는 라이브러리.

 

 

- 손실함수 관련 hyperparameters:

• learning rate : A scalar used to train a model via gradient descent. During each iteration, the gradient descent algorithm multiplies the learning rate by the gradient. The resulting product is called the gradient step.
• number of epochs : A full training pass over the entire dataset such that each example has been seen once. Thus, an epoch represents N/batch size training iterations, where N is the total number of examples.
• batch size : The number of examples in a batch. For example, the batch size of SGD is 1, while the batch size of a mini-batch is usually between 10 and 1000. Batch size is usually fixed during training and inference; however, TensorFlow does permit dynamic batch sizes.

 

- Summary of hyperparameter tuning

• Training loss should steadily decrease, steeply at first, and then more slowly until the slope of the curve reaches or approaches zero.
• If the training loss does not converge, train for more epochs.
• If the training loss decreases too slowly, increase the learning rate. Note that setting the learning rate too high may also prevent training loss from converging.
• If the training loss varies wildly (that is, the training loss jumps around), decrease the learning rate.
• Lowering the learning rate while increasing the number of epochs or the batch size is often a good combination.
• Setting the batch size to a very small batch number can also cause instability. First, try large batch size values. Then, decrease the batch size until you see degradation.
• For real-world datasets consisting of a very large number of examples, the entire dataset might not fit into memory. In such cases, you'll need to reduce the batch size to enable a batch to fit into memory.