IOS+Tensorflow for image recognition

Preface


Artificial intelligence, after the fire last year, Google opened its machine learning tool Tensorflow, now Tensorflow supports the mobile terminal, supported platforms include Android, iOS, Raspberry, Pi.

IOS+Tensorflow for image recognition

recently I also according to the TF installed on the computer in the tutorial, run at iOS Demo, the following figure is the Camera project screenshots, follow the tutorial can be downloaded if the trained Pb recognition can realize 1000 kinds of objects (see the Pb volume is estimated to be inception_v1, training images from ImageNet).

IOS+Tensorflow for image recognition
official renderings

But how do you add objects you want to recognize? The official gives a simple tutorial, but there are lots of holes, and this article shows you how to train your own pictures and run them on the iOS device.

Build training pictures


  • First of all, you need to prepare the pictures for the training. As follows:
IOS+Tensorflow for image recognition
images folder
  • There are all the categories in the tag. I have 9 categories here:
Fan Bingbing, Daniel Wu, Joe Chen, Jay Chou, TV, safety helmet, helmet, laptop, Changhong
  • Then, by compiling the tutorial build_image_data, you can run the picture data trained for the inception model.
To where to save # location the TFRecord data. OUTPUT_DIRECTORY=$HOME/my-custom-data/ build the preprocessing script. bazel build # # inception/build_image_data convert the data. bazel-bin/inception/build_image_data / --train_directory= ${TRAIN_DIR} / --validation_directory= ${VALIDATION_DIR} / --output_directory= "${OUTPUT_DIRECTORY}" / "${LABELS_FILE}" --labels_file= / --train_shards=128 / --validation_shards=24 / --num_threads=8

The resulting files are as follows:

IOS+Tensorflow for image recognition
images

Up to now, preparations for training and validation data have been completed.

Training model


Due to memory limitations, for better experience and use effects, it is best to reduce the accuracy of training the inception_v1 model, but I am lazy and simply follow the tutorial re training of the inception_v3 model.

Here I use the flowers_train in the tutorial, and then set num_classes to 9, because my picture comes in 9 categories. At the same time, the operation of model.ckpt inception_train.py records changed record every 10 times (the inception model is relatively large, I am single core CPU run, running time, if according to the original record 5000 times at a time then it will be about 83 hours, before the first record).

Set up, compile flowers_train, run, you can get the output of the checkpoints.

The model. Note that # Build we need to make sure the TensorFlow is ready to use this as this # before command will not build TensorFlow. bazel build inception/flowers_train Path to the downloaded Inception-v3 model. # "MODEL_PATH= ${INCEPTION_MODEL_DIR} /inception-v3/model.ckpt-157585" Directory the flowers data # where resides. FLOWERS_DATA_DIR=/tmp/flowers-data/ Directory to save the # where checkpoint and events files. TRAIN_DIR=/tmp/flowers_train/ Run the # fine-tuning on the flowers data set starting from the pre-trained Imagenet-v3 model. bazel-bin/inception/flowers_train # / --train_dir= "${TRAIN_DIR}" --data_dir= "${FLOWERS_DATA_DIR}" / "${MODEL_PATH}" / --pretrained_model_checkpoint_path= / --fine_tune=True / --initial_l Earning_rate=0.001 / --input_queue_memory_factor=1

Running screenshots and generated ckpt:

IOS+Tensorflow for image recognition
train
IOS+Tensorflow for image recognition
checkpoints

Generating PB files

In order to run on a iOS device, you must write the previously trained parameters into the graph and generate the file in protobuf format.
Google provides freeze_graph, which can be used directly to write graphs and arguments in pb.
there are 4 steps:

  1. Generate a calculation chart that performs the prediction, written in input.pb.
  2. Call freeze_graph, synthesize input.pb and ckpt.
  3. Call optimize_for_inference, reduce the operation, replace the calculation that cannot run in iOS, and generate inference.pb.
  4. Call quantize_graph, further optimize the operation diagram (this step does not start at first, and crashes most of the time after importing the iOS device) to generate rounded_graph.pb.

After completing these steps, you get the Pb file that you can import iOS to run:

IOS+Tensorflow for image recognition
PBS

Here is a shell script for me to assemble the entire process:

CD /Users/Jiao/Desktop/SecurityKeeper/tensorflow bazel-bin/tensorflow/python/tools/freeze_graph / --input_graph=/Users/Jiao/Desktop/TFProject/inception_v3/protobuf/input.pb --input_checkpoint=/Users/Jiao/Desktop/TFProject/inception_v3/checkpoints/ model.ckpt-80 / --output_node_names=inception_v3/logits/predictions --input_binary=False / --output_graph=/Users/Jiao/Desktop/TFProject/inception_v3/protobuf/frozen.pb bazel-bin/tensorflow/python/tools/optimize_for_inference / --input=/Users/Jiao/Desktop/ TFProject/inception_v3/protobuf/frozen.pb --output=/Users/Jiao/Desktop/TFProject/inception_v3/protobuf/inference.pb / --input_names=input --output_names=inception_v3/logits/predictions / --frozen_graph=True bazel-bin/tensorflow/tools/quantizati On/quantize_graph / --input=/Users/Jiao/Desktop/TFProject/inception_v3/protobuf/inference.pb / --output=/Users/Jiao/Desktop/TFProject/inception_v3/protobuf/rounded_graph.pb / --output_node_names=inception_v3/logits/predictions / --mode=weights_rounded

Configure the iOS parameter, run, Boom…

The front of the inception_v3 model, so the camera demo in some parameters need to change, the first is the const int wanted_input_width and const int wanted_input_height, have become 299, because the model input image size is 299*299, followed by the acquisition of image pixel normalization, directly divided by 255, normalized to [0, 1] interval.

/ / out_pixel[c] = (in_pixel[c] - input_mean) / input_std; out_pixel[c] = in_pixel[c] / 255;

Operation result:

IOS+Tensorflow for image recognition
TV set
IOS+Tensorflow for image recognition
safety helmet

epilogue

The above is the prediction of the parameters of the eightieth iteration, which is not ideal. But the whole process completed the model training and iOS devices running TF.

Reference:
, Getting, started, with, TensorFlow, on, iOS