How to Use Jupyter Notebook in 2020: A Beginner’s Tutorial

Tutorial: Advanced Jupyter Notebooks

First we know that the queue model is based on Poisson distribution. Here is three characteristics of Poisson distribution:

• The experiment consists of counting the number of events that will occur during a specific interval of time or in a specific distance, area, or volume.
• The probability that an event occurs in a given time, distance, area, or volume is the same.
• Each event is independent of all other events. For example, the number of people who arrive in the first hour is independent of the number who arrive in any other hour.

Little’s Theorem

N = λT

N=average number of customers

λ=Average arrival rate

T=Average sojourn(stay) time of a customer

Apply the Little’s Theorem to the Network Delay environment

We should also add some notation:

ρ: the line’s utilization factor(we can see that [latex]ρ=\frac{λ}{μ}[/latex] later)

X: average transmission time

[latex]ρ=λX[/latex]

now introduce the model:

M/M/1 Model

1. How does Resnet code in tensorflow/models be distributed?

High API Estimator

distributed_strategy in utils/misc

2. How to make codes in https://github.com/geifmany/cifar-vgg distributed?

Refer Tensorflow tutorial:

Set a distributed strategy and scope including model construction and model compile

However

VGG uses data augmentation which is in conflict with distribution!

In Keras tutorial, if we use fit_generator method, then we will meet this error:

fit_generator` is not supported for models compiled with tf.distribute.strategy.

Our Tensorflow version is 1.14

If we use ‘manual’ example in the official tutorial, then the training will become wield:

Use single GPU this is the std output:

Above is normal(though different from using fit_generator). Below is the distributed version using mirror strategy, it’s abnormal:

Distributed version stuck in the first epoch and the loss is high for a long time.

Solution:

This issue suggests using tf.data.Dataset.from_generator to deal with the generator.

Start with this issue:

https://github.com/tensorflow/tensorflow/issues/9400

This can explain why I use official resnet model and cannot do memory growth and set memory limit.

• Outline
• Scheduling
• Components:

Decides Server order

manage queue

• Why do we need one?
• What can scheduling disciplines do?
• Requirements of a scheduling discipline
• Ease of implementation
• Fairness

Fairness is global, scheduling(congestion avoidance) is local.

• Notion of Fairness
• Fundamental choices

Work-conserving and non-work-conserving

Degree of aggregation

• Scheduling disciplines

FIFO & other disciplines(SPT, SRPT), the performance among them

(SRPT process the remaining time, which means if I’m processing a package which still needs 5 min, then comes a package which only need 1 min, then I go to process the new package)

 

• The Conservation Law

scheduling is independent of the packet service time

[latex]\sum ρ_iq_i=constant[/latex]

[latex]ρ_i[/latex] mean utilization of connection i and [latex]q_i[/latex] mean waiting time of connection i

The average delay with FIFO is a tight lower bound for
work conserving and service time independent scheduling
disciplines

• Fairness

Jain’s index use equal share as the
objective:

[latex]f=\frac{(\sum_{i=1}^{N}x_i)^2}{(n\sum_{i=1}^{N}x_i^2)}[/latex]

• Max-Min Fairness
• General Process Sharing (GPS)

Conceptually, GPS serves packets as if they are
in separate logical queues, visiting each nonempty
queues in turn.

Generalized processor sharing assumes that traffic is fluid (infinitesimal packet sizes), and can be arbitrarily split.

How to emulate GPS as fair as possible, also efficient

• (Weighted) round robin

Different weights, fixed packet size

Different weights, variable size packets：normalize weights by mean packet size

Problems:

1. With variable size packets and different weights,
   need to know mean packet size in advance
2. Can be unfair for long periods of time