The uppers of shoes get folded over the toe portion every time a step is taken. These repeated foldings cause the finished grains to start cracking, resulting in loss of water resistance. To ensure that the leather uppers retain their water resistance for a reasonable period of use, the action of such foldings on them is evaluated by subjecting test specimens taken from the leather to a water penetration test using a penetrometer. In this apparatus the test specimens are held in form of a trough between two cylindrical grips. One of the grips is then repeatedly moved towards and away from the other along their common axis. The test specimens are kept immersed in a water bath up to a part of their periphery. Penetration is detected when an electric conductivity is established between the inside and outside surfaces of the test specimens. The time needed to cause penetration of water through the test specimens is determined and taken as an index of water resistance of the leather. The test can be further continued to determine the percent gain of weight of the specimen due to water absorption and also to determine the mass of water that is transmitted.