2 options
Balancing fit and complexity in learned representations / Maria Peifer.
- Format:
- Book
- Thesis/Dissertation
- Author/Creator:
- Peifer, Maria, author.
- Language:
- English
- Subjects (All):
- Artificial intelligence.
- Applied mathematics.
- Electrical engineering.
- Sparsity.
- Sample size.
- Accuracy.
- Datasets.
- Random variables.
- Communication.
- Classification.
- Approximation.
- Algorithms.
- Privacy.
- Objectives.
- Bias.
- Electrical and systems engineering--Penn dissertations.
- Penn dissertations--Electrical and systems engineering.
- Local Subjects:
- Artificial intelligence.
- Applied mathematics.
- Electrical engineering.
- Sparsity.
- Sample size.
- Accuracy.
- Datasets.
- Random variables.
- Communication.
- Classification.
- Approximation.
- Algorithms.
- Privacy.
- Objectives.
- Bias.
- Electrical and systems engineering--Penn dissertations.
- Penn dissertations--Electrical and systems engineering.
- Genre:
- Academic theses.
- Physical Description:
- 1 online resource (140 pages)
- Contained In:
- Dissertations Abstracts International 83-03B.
- Place of Publication:
- [Philadelphia, Pennsylvania] : University of Pennsylvania ; Ann Arbor : ProQuest Dissertations & Theses, 2021.
- Language Note:
- English
- System Details:
- Mode of access: World Wide Web.
- text file
- Summary:
- This dissertation is about learning representations of functions while restricting complexity. In machine learning, maximizing the fit and minimizing the complexity are two conflicting objectives. Common approaches to this problem involve solving a regularized empirical minimization problem, with a complexity measure regularizer and a regularizing parameter that controls the trade-off between the two objectives. The regularizing parameter has to be tuned by repeatedly solving the problem and does not have a straightforward interpretation. This work formulates the problem as a minimization of the complexity measure subject to the fit constraints. The issue of complexity is tackled in reproducing kernel Hilbert spaces (RKHSs) by introducing a novel integral representation of a family of RKHSs that allows arbitrarily placed kernels of different widths. The functional estimation problem is then written as a sparse functional problem, which despite being non-convex and infinite-dimensional can be solved in the dual domain. This problem achieves representations of lower complexity than traditional methods because it searches over a family of RKHS rather than a subspace of a single RKHS. The integral representation is used in a federated classification setting, in which a global model is trained from a federation of agents. This is possible because the dual optimal variables give information about the samples that are fundamental to the classification. Each agent, therefore, learns a local model and sends only the fundamental samples over the network. This creates a federated learning method that requires only one network communication. Its solution is proven to asymptotically converges to that of traditional classification. Next, a theory for constraint specification is established. An optimization problem with a constraint for each sample point can easily become infeasible if the constraints are too tight. In contrast, relaxing all constraints can cause the solution to not fit the data well. The constrained specification method relaxes the constraints until the marginal cost of changing a constraint is equal to the marginal complexity measure. This problem is proven to be feasible and solvable and shown empirically to be resilient to outliers and corrupted training data.
- Notes:
- Source: Dissertations Abstracts International, Volume: 83-03, Section: B.
- Advisors: Ribeiro, Alejandro; Committee members: Hassani, Hamed ; Daniilidis, Konstantinos ; Sadler, Brian M.
- Department: Electrical and Systems Engineering.
- Ph.D. University of Pennsylvania 2021.
- Local Notes:
- School code: 0175
- ISBN:
- 9798535590813
- Access Restriction:
- Restricted for use by site license.
- This item is not available from ProQuest Dissertations & Theses.
- This item must not be sold to any third party vendors.
The Penn Libraries is committed to describing library materials using current, accurate, and responsible language. If you discover outdated or inaccurate language, please fill out this feedback form to report it and suggest alternative language.