The rule for a dilation by a factor of k using the origin as the center of dilation, is:
[tex](x,y)\rightarrow(kx,ky)[/tex]
In the options A and B, additions and substractions are involved. Then, they cannot be the rule of a dilation about the origin.
In the options C and D, we can see that both are dilations about the origin. The factor used in the option C is 5/4, while the factor used in the option D is 0.9.
Nevertheless, notice that 5/4 is greater than 1 and 0.9 is smaller than 1.
Then, the dilation from option C would produce a bigger polygon, while the dilation from option D will produce a smaller polygon.
Since the polygon must be a smaller one, then the rule that could represent this dilation is:
[tex](x,y)\rightarrow(0.9x,0.9y)[/tex]
Which corresponds to option D.
