Climate change and rising global average temperatures across the year may strongly affect olive fruits’ development process and their oil yield and quality. There is therefore an urgency to take immediate actions to characterize the wide variability of cultivars in order to identify those with a stable response to high temperatures, particularly in areas like the west of Iran, which is characterized by a warm summer continental climate. The objective of this study is to investigate the process of fruit development and oil accumulation in response to high summer temperature conditions in a set of four Iranian olive cultivars (Shengeh, Roughani, Zard Aliabad, and Dezful) in comparison with four foreign olive cultivars (Konservolia, Sevillana, Manzanilla, and Mission) in seven various harvesting times (20 July, 5 and 20 August, 5 and 20 September, 6 and 21 October). The obtained results evidence a significant positive correlation between fruit dry matter and oil content. High temperatures reduced the oil and dry matter accumulation in the second half of the summer, with severe thermal conditions adversely affecting oil synthesis. Paramount variations were observed among the cultivars regarding oil accumulation, dry matter, and pomological attributes. All of them showed the highest oil content at the last harvest. Among all analyzed varieties, Roughani showed the highest tolerance and adaptive capacity to high temperatures as it accumulated the greatest amount of dry matter as well as oil content in all of the harvesting times, demonstrating a positive correlation between these two traits. Although Shengeh showed the lowest oil content on a dry and fresh weight basis at the first harvesting time, this cultivar generally presented higher fruit development attributes than the other cultivars, highlighting that it benefits from a high temperature.