The image size is taking effect for all items in the portfolio grid as verified by me. The different sizes of images you see is due to how WordPress itself crops the images based on the settings specified by you in Settings->Media in WordPress admin. WordPress does not hard crop the images like some of the other tools. It still maintains the same aspect ratio of the original images. If you need images to be hard cropped, pls see below article on how to accomplish the same –
http://www.wpbeginner.com/wp-tutorials/how-to-create-additional-image-sizes-in-wordpress/
Additionally you can look for some plugins which may help you accomplish the same.