Results

Found 8 articles

Clear filters

A Prescriptive Analytics Framework for Optimal Policy Deployment Using Heterogeneous Treatment Effects

2021 | Management Information Systems Quarterly | Citations: 8
Authors: McFowland, Edward; Gangarapu, Sandeep; Bapna, Ravi; Sun, Tianshu

Abstract: We define a prescriptive analytics framework that addresses the needs of a const ... Expand

Abstract: We define a prescriptive analytics framework that addresses the needs of a constrained decision-maker facing, ex ante, unknown costs and benefits of multiple policy levers. The framework is general in nature and can be deployed in any utility-maximizing context, public or private. It relies on randomized field experiments for causal inference, machine learning for estimating heterogeneous treatment effects, and on the optimization of an integer linear program for converting predictions into decisions. The net result is the discovery of individual-level targeting of policy interventions to maximize overall utility under a budget constraint. The framework is set in the context of the four pillars of analytics and is especially valuable for companies that already have an existing practice of running A/B tests. The key contribution of this work is to develop and operationalize a framework to exploit both within- and between-treatment arm heterogeneity in the utility response function in order to derive benefits from future (optimized) prescriptions. We demonstrate the value of this framework as compared to benchmark practices—i.e., the use of the average treatment effect, uplift modeling, as well as an extension to contextual bandits—in two different settings. Unlike these standard approaches, our framework is able to recognize, adapt to, and exploit the (potential) presence of different subpopulations that experience varying costs and benefits within a treatment arm while also exhibiting differential costs and benefits across treatment arms. As a result, we find a targeting strategy that produces an order of magnitude improvement in expected total utility for the case where significant within- and between-treatment arm heterogeneity exists. Collapse

Semantic filters: multi-armed bandit

Topics: marketing management decision making economic surplus IS economics analytical information system

Methods: experiment computational algorithm machine learning causality analysis experimental group
Optimal pricing in e-commerce based on sparse and noisy data
PDF
2018 | Decision Support Systems | Citations: 0
Authors: Bauer, Josef; Jannach, Dietmar

Abstract: In today's transparent markets, e-commerce providers often have to adjust their ... Expand

Abstract: In today's transparent markets, e-commerce providers often have to adjust their prices within short time intervals, e.g., to take frequently changing prices of competitors into account. Automating this task of determining an “optimal” price (e.g., in terms of profit or revenue) with a learning-based approach can however be challenging. Often, only few data points are available, making it difficult to reliably detect the relationships between a given price and the resulting revenue or profit. In this paper, we propose a novel machine-learning based framework for estimating optimal prices under such constraints. The framework is generic in terms of the optimality criterion and can be customized in different ways. At its core, it implements a novel algorithm based on Bayesian inference combined with bootstrap-based confidence estimation and kernel regression. Simulation experiments show that our method is favorable over existing dynamic pricing strategies. Furthermore, the method led to a significant increase in profit and revenue in a real-world evaluation. Collapse

Semantic filters: multi-armed bandit

Topics: price management dynamic pricing electronic commerce online learning system

Methods: computational algorithm simulation experiment decision tree classification multi-armed bandit
Learning to Shift Thermostatically Controlled Loads
PDF
2017 | HICSS | Citations: 0
Authors: Lesage-Landry, Antoine; Taylor, Joshua A.

Abstract: Demand response is a key mechanism for accommodating renewable power in the elec ... Expand

Abstract: Demand response is a key mechanism for accommodating renewable power in the electric grid. Models of loads in demand response programs are typically assumed to be known a priori, leaving the load aggregator the task of choosing the best command. However, accurate load models are often hard to obtain. To address this problem, we propose an online learning algorithm that performs demand response while learning the model of an aggregation of thermostatically controlled loads. Specifically, we combine an adversarial multi-armed bandit framework with a standard formulation of load-shifting. We develop an Exp3-like algorithm to solve the learning problems. Numerical examples based on Ontario load data confirm that the algorithm achieves sub-linear regret and performs within 1% of the ideal case when the load is perfectly known. Collapse

Semantic filters: multi-armed bandit

Topics: online learning system

Methods: computational algorithm simulation multi-armed bandit optimization model
From roles to standards: a dynamic maintenance approach using incentives
PDF
2015 | Information Systems Frontiers | Citations: 0
Authors: Hermoso, Ramón; Lopes Cardoso, Henrique; Fasli, Maria

Abstract: Social coordination has been addressed in multi-agent systems, making use of con ... Expand

Abstract: Social coordination has been addressed in multi-agent systems, making use of concepts such as institutions, norms, commitments, conventions, roles, or trust. In this paper, we argue the need to tackle open and dynamic environments with yet another concept: the notion of a standard, seen as a measurable and non-committing expectation. Not much work has been done in the field of multi-agent systems addressing the evolving nature of roles, especially in open systems, in which changes in the population bring about changes in the expectations generated from roles. Using standards measured from roles as the focus of attention, we propose an incentive-based mechanism to maintain roles over time. This approach is put in contrast with reorganization, which is needed when incentives are not cost-effective. Different search algorithms are proposed to illustrate incentive-based maintenance. Some empirical results are shown based on the principal-agent model from economics. Collapse

Semantic filters: multi-armed bandit

Topics: conversational agent

Methods: computational algorithm experiment reinforcement learning simulation optimization model
Online learning for auction mechanism in bandit setting
PDF
2013 | Decision Support Systems | Citations: 1
Authors: He, Di; Chen, Wei; Wang, Liwei; Liu, Tie-Yan

Abstract: This paper is concerned with online learning of the optimal auction mechanism fo ... Expand

Abstract: This paper is concerned with online learning of the optimal auction mechanism for sponsored search in a bandit setting. Previous works take the click-through rates of ads to be fixed and known to the search engine and use this information to design optimal auction mechanism. However, the assumption is not practical since ads can only receive clicks when they are shown to users. To tackle this problem, we propose to use online learning for auction mechanism design. To be specific, this task corresponds to a new type of bandit problem, which we call the armed bandit problem with shared information (AB-SI). In the AB-SI problem, the arm space (corresponding to the parameter space of the auction mechanism which can be discrete or continuous) is partitioned into a finite number of clusters (corresponding to the finite number of rankings of the ads), and the arms in the same cluster share the explored information (i.e., the click-through rates of the ads in the same ranked list) when any arm from the cluster is pulled. We propose two upper-confidence-bound algorithms called UCB-SI1 and UCB-SI2 to tackle this new problem in discrete-armed bandit and continuum-armed bandit setting respectively. We show that when the total number of arms is finite, the regret bound obtained by UCB-SI1 algorithm is tighter than the classical UCB1 algorithm. In the continuum-armed bandit setting, our proposed UCB-SI2 algorithm can handle a larger classes of reward function and achieve a regret bound of O(T2/3(dlnT)1/3), where d is the pseudo dimension for the real-valued reward function class. Experimental results show that the proposed algorithms can significantly outperform several classical online learning methods on synthetic data. Collapse

Semantic filters: multi-armed bandit

Topics: online learning system search engine search engine marketing price management advertising management

Methods: computational algorithm multi-armed bandit experiment supervised learning optimization model
Efficiently gathering information in costly domains
PDF
2013 | Decision Support Systems | Citations: 0
Authors: Reches, Shulamit; Gal, Ya'akov (Kobi); Kraus, Sarit

Abstract: This paper proposes a novel technique for allocating information gathering actio ... Expand

Abstract: This paper proposes a novel technique for allocating information gathering actions in settings where agents need to choose among several alternatives, each of which provides a stochastic outcome to the agent. Samples of these outcomes are available to agents prior to making decisions and obtaining further samples is associated with a cost. The paper formalizes the task of choosing the optimal sequence of information gathering actions in such settings and establishes it to be NP-Hard. It suggests a novel estimation technique for the optimal number of samples to obtain for each of the alternatives. The approach takes into account the trade-offs associated with using prior samples to choose the best alternative and paying to obtain additional samples. This technique is evaluated empirically in several different settings using real data. Results show that our approach was able to significantly outperform alternative algorithms from the literature for allocating information gathering actions in similar types of settings. These results demonstrate the efficacy of our approach as an efficient, tractable technique for deciding how to acquire information when agents make decisions under uncertain conditions. Collapse

Semantic filters: multi-armed bandit

Topics: database system artificial intelligence

Methods: computational algorithm descriptive statistic parametric test Student's t-test simulation
News Recommender Systems with Feedback

2012 | International Conference on Information Systems | Citations: 0
Authors: Prawesh, Shankar; Padmanabhan, Balaji

Abstract: The focus of present research is widely used news recommendation techniques such ... Expand

Abstract: The focus of present research is widely used news recommendation techniques such as “most popular” or “most e-mailed”. In this paper we have introduced an alternative way of recommendation based on feedback. Various notable properties of the feedback based recommendation technique have been also discussed. Through simulation model we show that the recommendation technique used in the present research allows implementers to have a flexibility to make a balance between accuracy and distortion. Analytical results have been established in a special case of two articles using the formulation based on generalized urn models. Finally, we show that news recommender systems can be also studied through two armed bandit algorithms. Collapse

Semantic filters: multi-armed bandit

Topics: recommender system website advertising management Digg

Methods: simulation computational algorithm literature sample machine learning theoretical contribution
Exploitation vs. exploration: choosing a supplier in an environment of incomplete information
PDF
2004 | Decision Support Systems | Citations: 0
Authors: Azoulay-Schwartz, Rina; Kraus, Sarit; Wilkenfeld, Jonathan

Abstract: An agent operating in the real world must often choose between maximizing its ex ... Expand

Abstract: An agent operating in the real world must often choose between maximizing its expected utility according to its current knowledge about the world and trying to learn more about the world, since this may improve its future gains. This problem is known as the trade-off between exploitation and exploration. In this research, we consider this problem in the context of electronic commerce. An agent intends to buy a particular product (goods or service). There are several potential suppliers of this product, but they differ in their quality and in the price charged. The buyer cannot observe the average quality of each product, but he has some knowledge about the quality of previous goods purchased from the suppliers. On the one hand, the buyer is motivated to buy the goods from the supplier with the highest expected product quality, deducting the product price. However, when buying from a lesser known supplier, the buyer can learn about its quality and this can help him in the future, when he will purchase more products of this type. We show the similarity of the suppliers problem to the k-armed bandit problem, and we suggest solving the suppliers problem by evaluating Gittins indices and choosing the supplier with the optimal index. We demonstrate how Gittins indices are calculated in real world situations, where deals of different magnitudes may exist, and where product prices may vary. Finally, we consider the existence of suppliers with no history and suggest how the original Gittins indices can be adapted in order to consider this extension. Collapse

Semantic filters: multi-armed bandit

Topics: database system electronic commerce experience good switching cost

Methods: descriptive statistic simulation multi-armed bandit Gittins index literature study

Page 1 of 1

AIS11

Journals

Conferences

Semantic Filter

Found 8 articles

A Prescriptive Analytics Framework for Optimal Policy Deployment Using Heterogeneous Treatment Effects

Optimal pricing in e-commerce based on sparse and noisy data

Learning to Shift Thermostatically Controlled Loads

From roles to standards: a dynamic maintenance approach using incentives

Online learning for auction mechanism in bandit setting

Efficiently gathering information in costly domains

News Recommender Systems with Feedback

Exploitation vs. exploration: choosing a supplier in an environment of incomplete information