According to Joseph, his older brother Hyrum joined the Presbyterian church along with his mother as a result of the revival. Willard [p.17]Chase, a neighbor, mentioned that in 1825 Hyrum asked to borrow his seer stone. Though reluctant to let the stone go, Chase said he honored Hyrum’s request because Hyrum “had made a profession of religion” and Chase felt he could now be trusted to return it. ---Marquardt & Walters, Inventing Mormonism, (pp.16-17)The story both shows that Hyrum was not implicitly trustworthy; that their religious conversion experience bestowed a sense of trustworthiness on them that the community was not yet willing to accord them on their own (how hard they worked, be damned); and that the trust they were now allotted was dependent on fulfilling the twin conditions of staying religious and behaving in a trustworthy fashion.
When Hyrum later refused to return the stone and dropped out of the Presbyterian church, both of these conditions were violated and the Smiths became the scum the affidavits make them out to be.
Daniel Peterson has pointed to the following quote by Joseph's brother William (in: Deseret Evening News, 20 January 1894, p. 11; compare the verdict of Richard L. Bushman, Joseph Smith and the Beginnings of Mormonism (Urbana: University of Illinois, 1984), 190.)
We never knew we were bad folks until Joseph told his vision. We were considered respectable til then, but at once people began to circulate falsehoods and stories in wonderful ways. (p.11)