Tanisha
I'll split my answer in two sections.
Input preprocessing:
Check how inputs are being pre-processed on local computer. You are not applying any optimization parameters, which means that you have to pass images that are already preprocessed.
Example, if during training you take BGR image and scale it to 0-1 range by dividing with 255, you will have to feed such images over XLinkIn based on the flags you use during conversion. Alternatively, you could add "--scale 255" to optimizer_params, and then you could feed images with 0-255 range, as network will first divide them by 255 before running the inference. Note this is just an example and it depends on how your pre-processing actually looks like.
Results:
It looks like you are running softmax directly on the output that you get from the network. This is wrong. You first need to take the outputs and reshape them to the same shape as locally. I'd modify post-processing as such:
output = np.array(output)
output = output.reshape((5, nn_shape, nn_shape))
output = np.transpose(output, (1,2,0))
test_pred = np.argmax(output, axis=2)
test_prob = softmax(output)
Not sure how you softmax looks like, but you need to make sure it works with axis=2. For classes, argmax is enough, softmax is only needed if you want probability of each class as well.